FraudBench: A Benchmark for Web Fraud Attacks Against LLM-Driven Agents

ICLR 2026 Conference Submission16731 Authors

Keywords: agent, web fraud attack, benchmark

TL;DR: The first benchmark that evaluate agent's vulnerabilities in the face of web fraud attacks, a novel kind of attack against agents

Abstract: LLM-driven agents are being severely threatened by web fraud attacks, which aim to induce agents to visit malicious websites. Upon success, attackers can use these websites to launch numerous subsequent attacks, which dramatically enlarges the attack surface. However, there have not been systematic benchmarks specifically designed for this newly emerging threat. To this end, this paper proposes FraudBench, the first dedicated benchmark of web fraud attacks. FraudBench contains over 61,845 attack instances across 10 distinct scenarios, 7 categories of real-world malicious websites. Experiments using 11 popular LLMs reveal that web fraud attacks have high attack success rates on them. Besides, we also comprehensively analyze the critical factors that can influence the attack success rate observed in the experiments. Our work provides in-depth insight into web fraud attacks for the first time and demonstrates the urgency of paying attention to agent security when handling web links.

Primary Area: datasets and benchmarks

Submission Number: 16731