VAGUE‑Gate: Plug‑and‑Play Local‑Privacy Shield for Retrieval‑Augmented Generation

ACL ARR 2025 July Submission1467 Authors

Abstract:

Retrieval‑augmented generation (RAG) still \emph{forwards} raw passages to large‑language models, so private facts slip through. Prior defences are either (i) \textbf{heavyweight}—full DP training that is impractical for today’s 70 B‑parameter models—or (ii) \textbf{over‑zealous}—blanket redaction of every named entity, which slashes answer quality.
We introduce \textbf{\textsc{VAGUE‑Gate}}, a lightweight, \emph{locally} differentially‑private gate deployable in front of \emph{any} RAG system. A precision pass drops low‑utility tokens under a user budget~$\varepsilon$, then up to $k(\varepsilon)$ high‑temperature paraphrase passes further cloud residual cues; post‑processing guarantees preserve the same $\varepsilon$‑LDP bound.

To measure both privacy and utility, we release \textsc{PrivRAG} (3k blended‑sensitivity QA pairs) and two new metrics: a lexical Information‑Leakage Score and an LLM‑as‑Judge score. Across eight pipelines and four SOTA LLMs, \textsc{VAGUE‑Gate} at $\varepsilon=0.3$ lowers lexical leakage by \textbf{70 \%} and semantic leakage by \textbf{1.8} points (1–5 scale) while retaining \textbf{91\%} of Plain‑RAG faithfulness with only a 240ms latency overhead.
All code, data, and prompts are publicly released.\footnote{\url{https://github.com/LLMGreen/LDP_RAG}}

Paper Type: Long

Research Area: Efficient/Low-Resource Methods for NLP

Research Area Keywords: retrieval-augmented generation, security/privacy, efficient models, evaluation methodologies, corpus creation, benchmarking

Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency, Data resources, Data analysis

Languages Studied: English

Reassignment Request Area Chair: This is not a resubmission

Reassignment Request Reviewers: This is not a resubmission

A1 Limitations Section: This paper has a limitations section.

A2 Potential Risks: N/A

B Use Or Create Scientific Artifacts: Yes

B1 Cite Creators Of Artifacts: Yes

B1 Elaboration: Model details in our appendix

B2 Discuss The License For Artifacts: Yes

B2 Elaboration: \subsection*{Consistency of Artifact Use With Intended Purpose} in Appendix

B3 Artifact Use Consistent With Intended Use: Yes

B3 Elaboration: \subsection*{Consistency of Artifact Use With Intended Purpose} in Appendix

B4 Data Contains Personally Identifying Info Or Offensive Content: No

B4 Elaboration: There is no offensive content in our paper.

B5 Documentation Of Artifacts: Yes

B5 Elaboration: Almost in all sections (1,3,4,5)

B6 Statistics For Data: Yes

B6 Elaboration: Dataset Section (section 3)

C Computational Experiments: Yes

C1 Model Size And Budget: Yes

C1 Elaboration: Dataset section in Appendix, and section 4 and 5

C2 Experimental Setup And Hyperparameters: Yes

C2 Elaboration: Experiment section (number 5)

C3 Descriptive Statistics: Yes

C3 Elaboration: Section 5

C4 Parameters For Packages: Yes

C4 Elaboration: \subsection*{Software Packages and Parameter Settings} in Appendix

D Human Subjects Including Annotators: No

D1 Instructions Given To Participants: N/A

D2 Recruitment And Payment: N/A

D3 Data Consent: N/A

D4 Ethics Review Board Approval: N/A

D5 Characteristics Of Annotators: N/A

E Ai Assistants In Research Or Writing: Yes

E1 Information About Use Of Ai Assistants: Yes

E1 Elaboration: \subsection*{Information About Use of AI Assistants} in Appendix

Author Submission Checklist: yes

Submission Number: 1467