All Your LLMs Belong to Us: Experiments with a New Extortion Phishing Dataset

Fatima Zahra Qachfar, Rakesh M. Verma

Abstract: In the last decade, there has been a dramatic rise in phishing emails including business email compromise, and extortion attacks. Ransomware and blackmail are examples of extortion threats, where attackers force victims to follow orders, send money, or share sensitive information. To our knowledge, no phishing dataset that includes text-based extortion attacks has been made available to the security research community. To address this problem, we present “TExtPhish” consisting of a sentence-level subset and a full-body email-level subset that can be used for multiple classification and regression tasks. We also provide another challenging subset with homograph text perturbations to address a specific Unicode NLP attack targeting primarily LLMs, causing them to hallucinate and significantly degrade their performance. We show this by conducting multiple experiments including extortion classification, sentiment analysis, and language identification. Our findings indicate that DistilBERT is most susceptible to homograph attacks at sentence level, resulting in a \(94.9\%\) decrease in F1-score while DeBERTa’s performance decreased \(94.1\%\) at email level.