One Cloud Is Enough to Eclipse All the Sun! Retrieval-Augmented Generation Attack by Event Element Differentiation

ACL ARR 2025 February Submission3279 Authors

Abstract:

The combination of large language model (LLM) and retrieval-augmented generation (RAG) frameworks is currently the mainstream approach for the LLM-based application. Yet this reliance on external data introduces new security risks, particularly corpus poisoning, which involves injecting malicious records into the knowledge base to manipulate the retriever in the RAG process. The key to successful corpus poisoning lies in ensuring that the malicious records are both retrievable and sufficiently stealthy to evade detection. However, existing methods struggle to achieve these two objectives simultaneously.

In this paper, we propose a stealthy corpus poisoning approach for attacking RAG-LLM systems, specifically targeting event elements—such as place, person, and time—to mislead the LLM. These event elements are fundamental components of human cognition and understanding of events, as they define the "who", "where" and ``when'' of occurrences, shaping how individuals perceive and interpret information. By subtly poisoning these critical elements in the retrieved corpus, attackers can manipulate the LLM's outputs in ways that are both impactful and difficult to detect. The experimental results show that our approach can have more than 70% attack success rate. And the samples generated by our approach exhibit significantly enhanced resistance to identification by the adversarial sample detection technique. This reveals that the new security risks under RAG paradigm need to be paid enough attention and the corresponding defense strategy should be proposed urgently.

Paper Type: Long

Research Area: Language Modeling

Research Area Keywords: retrieval-augmented models,security and privacy

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 3279