Defending Against Social Engineering Attacks in the Age of LLMs
Abstract: The proliferation of Large Language Models (LLMs) poses challenges in detecting and mitigating digital deception, as these models can emulate human conversational patterns and facilitate chat-based social engineering (CSE) attacks. This study investigates the dual capabilities of LLMs as both facilitators and defenders against CSE threats. We develop a novel dataset, $\textbf{SEConvo}$, simulating CSE scenarios in academic and recruitment contexts, and designed to examine how LLMs can be exploited in these situations. Our findings reveal that, while off-the-shelf LLMs generate high-quality CSE content, their detection capabilities are suboptimal, leading to increased operational costs for defense. In response, we propose $\textbf{ConvoSentinel}$, a modular defense pipeline that improves detection at both the message and the conversation levels, offering enhanced adaptability and cost-effectiveness. The retrieval-augmented module in $\textbf{ConvoSentinel}$ identifies malicious intent by comparing messages to a database of similar conversations, enhancing CSE detection at all stages. Our study highlights the need for advanced strategies to leverage LLMs in cybersecurity. Our code and data are available at this anonymous repo link: https://anonymous.4open.science/r/ConvoSentinel_Anonymous-1E1D/README.md.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: Social Engineering Defense, Digital Deception Detection, Cybersecurity, Large Language Models, Malicious Intent Detection, Conversation Analysis
Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models, Data resources
Languages Studied: English
Submission Number: 1479
Loading