EVADE-Bench: Multimodal Benchmark for Evasive Content Detection in E-Commerce Applications

Ancheng Xu, Zhihao Yang, Jingpeng Li, Guanghu Yuan, Longze Chen, Liang Yan, Jiehui Zhou, Zhen Qin, Hengyu Chang, Hamid Alinejad-Rokny, Min Yang

Keywords: E-commerce, Evasive Content Detection, Benchmark, Large Language Models, Vision Language Models, Chinese

TL;DR: Our paper introduces EVADE-Bench, a benchmark for evaluating how LLMs and VLMs handle evasive content in e-commerce, focusing on detecting misleading ads and comparing the performance of open-source and closed-source models.

Abstract: E-commerce platforms increasingly rely on Large Language Models (LLMs) and Vision–Language Models (VLMs) to detect illicit or misleading product content. However, these models remain vulnerable to evasive content: inputs (text or images) that superficially comply with platform policies while covertly conveying prohibited claims. Unlike traditional adversarial attacks that induce overt failures, evasive content exploits ambiguity and context, making it far harder to detect. Existing robustness benchmarks provide little guidance for this high-stakes, real-world challenge. We introduce EVADE-Bench, the first expert-curated, Chinese, multimodal benchmark specifically designed to evaluate foundation models on evasive content detection in e-commerce. The dataset contains 2,833 annotated text samples and 13,961 annotated images spanning six categories, including body shaping, height growth, health supplements, and others. Two complementary tasks assess distinct capabilities: Single-Violation, which probes fine-grained reasoning under short prompts, and All-in-One, which tests long-context reasoning by merging overlapping policy rules into unified instructions. Our benchmarking of 26 mainstream LLMs and VLMs reveals that even state-of-the-art models frequently misclassify evasive samples. By releasing EVADE-Bench, we provide the first rigorous standard for evaluating evasive-content detection, expose fundamental limitations in current multimodal reasoning, and lay the groundwork for safer and more transparent content moderation systems in e-commerce.

Primary Area: datasets and benchmarks

Submission Number: 8920