E$^2$-RAG: Towards Editable Efficient RAG by Editing Compressed KV Caches

Tongxu Luo, Wenyu Du, HanWen Hao, Min Zhang, Hao Yang, Benyou Wang

Keywords: Retrieval Augmented Generation

TL;DR: E$^2$-RAG efficiently edits compressed KV caches for knowledge updates, achieving 40x faster editing and 3x faster generation than standard RAG with minimal performance loss.

Abstract:

Retrieval-Augmented Generation (RAG) demonstrates remarkable capabilities for enhancing the performance of Large Language Models (LLMs) by integrating external knowledge.
Standard RAG introduces additional computations due to the extra retrieved context.
To improve efficiency, recent studies propose compressing chunk tokens into compact forms, such as key-value (KV) caches.
However, maintaining these compressed KV caches in an updated state presents a significant challenge, undermining the primary goal of RAG: acquiring up-to-date knowledge.
In this work, we propose **E$^{2}$-RAG**, the first **E**ditable **E**fficient-**RAG** method designed to efficiently edit compressed KV caches for knowledge updates.
E$^2$-RAG features an encoder-decoder architecture similar to efficient RAG methods, along with an additional editor.
The encoder-decoder compresses chunk tokens into KV caches and generates responses.
The editor takes old KV caches and new knowledge tokens as inputs, enabling efficient updates to the KV caches.
To formalize knowledge updating, we define three operations: INSERT, DELETE, and UPDATE.
We create three sets of datasets for each operation.
Through extensive experiments, E$^2$-RAG achieves nearly **40x faster** editing compared to recomputing KV caches while maintaining **3x faster** generation efficiency than standard RAG, with a performance downgrade of 1%-5%.
We also conduct various ablation studies, including multi-turn editing, multi-chunk capability, and knowledge conflicts, to explore the capabilities of E$^2$-RAG.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the COLM Code of Ethics on https://colmweb.org/CoE.html

Author Guide: I certify that this submission complies with the submission instructions as described on https://colmweb.org/AuthorGuide.html

Submission Number: 1944