DMA: Enhancing Retrieval-Augmented Generation with Adaptive Human Feedback

Yu Bai, Yukai Miao, Li Chen, Dawei Wang, Dan Li, Yanyu Ren, Tianfeng Liu, Fei Long, Rundi Zhai, XieHongtao, CaiXuhui, YangCe

Keywords: Retrieval-Augmented Generation (RAG), Online Learning

Abstract: Retrieval-augmented generation (RAG) systems typically rely on static retrieval methods, limiting their adaptability to dynamic environments. In this paper, we propose a novel online learning framework called Dynamic Memory Alignment (DMA), designed specifically to enhance retrieval performance and content generation in RAG through adaptive incorporation of multi-level human feedback. DMA systematically integrates real-time feedback signals at document, list, and response levels, effectively adjusting memory management strategies to optimize relevance and adaptability in online interactive environments. Extensive evaluations demonstrate DMA’s competitive foundational retrieval performance across multiple standard knowledge-intensive benchmarks. Notably, DMA achieves significant advantages on datasets reflecting natural conversational interactions (TriviaQA, HotpotQA), highlighting its particular suitability for online GenAI dialogue applications. Moreover, a multi-month industrial deployment demonstrates that DMA substantially improves user engagement in real-world applications. These results underscore DMA’s ability to maintain robust foundational retrieval capabilities while excelling at dynamic, real-time adaptation in interactive online environments.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 8481