LLM and User Feedback Based Contarstive Learning Improves Retrieval-Augmented Generation When Question and Answer Domains Shift

ACL ARR 2025 May Submission7587 Authors

Abstract: Retrieval-Augmented Generation (RAG) attempts to mitigate the issue of outdated knowledge and hallucinations in large language models (LLMs) by retrieving real-time information for LLMs. Nevertheless, we observe that the domain of user questions undergoes rapid changes over time, resulting in a significant decrease in RAG performance. Meanwhile, existing methods either overlook the feedback present in the workflow or fail to fully utilize them to improve the RAG system. To this end, we propose a method that utilizes both LLM and User Feedback (LUF) to improve RAG performance with shifts in question domains and answer domains. With the framework designed to automatically extract diverse feedback signals from both LLM and user within the existing workflow, LUF can adjust to variations in questions and user preferences through updates to the retriever and document database, guided by three complementary training objectives derived from feedback-all without explicit annotations. Experiments on two tasks demonstrate that LUF significantly improves the accuracy of the retriever and the responses of the LLM. Compared to baselines, LUF provides more accurate responses aligned with different user preferences.

Paper Type: Long

Research Area: Dialogue and Interactive Systems

Research Area Keywords: Retrieval, Retrieval-augmented generation, Large language models

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 7587