OpenRAG: Optimizing RAG End-to-End via In-Context Retrieval Learning

Jiawei Zhou, Lei Chen

Keywords: Retrieval-augmented generation; RAG; LLM

TL;DR: A RAG framework that tunes a retriever end-to-end for downstream wild tasks.

Abstract: Retrieval-augmented generation (RAG) has become a prominent paradigm for enhancing generative models by leveraging rapidly expanding downstream data. However, exisiting RAG frameworks typically use off-the-shelf retrievers with large language models (LLMs) without joint training. In this paper, we provide analysis and empirical evidence showing that the relevance learned from conventional information retrieval settings do not consistently align with the needs of RAG applications. To bridge this gap, we introduce OpenRAG, a RAG framework that is optimized end-to-end by tuning the retriever to capture in-context, open-ended relevance, enabling adaptation to the diverse and evolving needs. Extensive experiments across a wide range of tasks demonstrate that OpenRAG, by tuning a retriever end-to-end, leads to a consistent improvement of 4.0% over the original retriever, consistently outperforming existing state-of-the-art retrievers by 2.1%. Additionally, our results show that for certain tasks, a 0.2B retriever tuned end-to-end can achieve improvements surpassing those of RAG-oriented or instruction-tuned 8B LLMs, underscoring the cost-effectiveness of our approach for enhancing RAG systems.

Submission Number: 15