VeriDark: A Large-Scale Benchmark for Authorship Verification on the Dark Web

Andrei Manolache, Florin Brad, Antonio Barbalau, Radu Tudor Ionescu, Marius Popescu

Keywords: authorship verification, dark web, authorship identification, authorship analysis, forensics, cybersecurity, natural language processing

TL;DR: We release datasets for authorship verification and identification collected from two DarkWeb forums and a DarkWeb-related subreddit

Abstract: The Dark Web represents a hotbed for illicit activity, where users communicate on different market forums in order to exchange goods and services. Law enforcement agencies benefit from forensic tools that perform authorship analysis, in order to identify and profile users based on their textual content. However, authorship analysis has been traditionally studied using corpora featuring literary texts such as fragments from novels or fan fiction, which may not be suitable in a cybercrime context. Moreover, the few works that employ authorship analysis tools for cybercrime prevention usually employ ad-hoc experimental setups and datasets. To address these issues, we release VeriDark: a benchmark comprised of three large scale authorship verification datasets and one authorship identification dataset obtained from user activity from either Dark Web related Reddit communities or popular illicit Dark Web market forums. We evaluate competitive NLP baselines on the three datasets and perform an analysis of the predictions to better understand the limitations of such approaches. We make the datasets and baselines publicly available at https://github.com/bit-ml/VeriDark .

Author Statement: Yes

URL: https://github.com/bit-ml/VeriDark/tree/master/datasets

Dataset Url:

The datasets are listed at this github page https://github.com/bit-ml/VeriDark/tree/master/datasets. The Zenodo links are available at the following links:
- SilkRoad1 authorship verification dataset: https://zenodo.org/record/6998371
- Agora authorship verification dataset: https://zenodo.org/record/7018853
- DarkReddit+ authorship verification dataset: https://zenodo.org/record/6998375
- DarkReddit+ author identification dataset: https://zenodo.org/record/6998363

Dataset Embargo: N/A

License: Creative Commons Attribution 4.0 International (CC-BY-4.0)

Contribution Process Agreement: Yes

In Person Attendance: Yes

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2207.03477/code)