Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

Anonymous

Abstract: Retrieval-Augmented Generation (RAG) improves Language Models (LMs) by incorporating external knowledge at test time to enable customized adaptation. We study the risk of datastore leakage in Retrieval-In-Context based RAG systems. We show that an adversary can exploit LMs' instruction-following capabilities to easily extract text data verbatim from the datastore of RAG systems built with instruction-tuned LMs via prompt injection. The vulnerability exists for a wide range of modern LMs that span Llama2, Mistral/Mixtral, Vicuna, SOLAR, WizardLM, Qwen1.5, and Platypus2, and the exploitability exacerbates as the model size scales up. Extending our study to production RAG models GPTs, we design an attack that can cause datastore leakage with a 100% success rate on 25 randomly selected customized GPTs with at most 2 queries, and we extract text data verbatim at a rate of 41% from a book of 77,000 words and 3% from a corpus of 1,569,000 words by prompting the GPTs with only 100 queries generated by themselves.

Paper Type: short

Research Area: Interpretability and Analysis of Models for NLP

Contribution Types: Model analysis & interpretability, Data analysis

Languages Studied: English

Preprint Status: We are considering releasing a non-anonymous preprint in the next two months (i.e., during the reviewing process).

A1: yes

A1 Elaboration For Yes Or No: Limitations

A2: yes

A2 Elaboration For Yes Or No: Ethical Considerations

A3: yes

A3 Elaboration For Yes Or No: Abstract,1

B: yes

B1: yes

B1 Elaboration For Yes Or No: 3,4

B2: yes

B2 Elaboration For Yes Or No: 3,4

B3: yes

B3 Elaboration For Yes Or No: 3,4

B4: yes

B4 Elaboration For Yes Or No: 3,4

B5: yes

B5 Elaboration For Yes Or No: 3,4

B6: yes

B6 Elaboration For Yes Or No: 3,4

C: yes

C1: yes

C1 Elaboration For Yes Or No: Appendix B

C2: yes

C2 Elaboration For Yes Or No: Appendix B

C3: yes

C3 Elaboration For Yes Or No: Appendix B

C4: yes

C4 Elaboration For Yes Or No: Appendix B

D: no

E: yes

E1: yes

E1 Elaboration For Yes Or No: 3,4