Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
Abstract: Large Language Models (LLMs) showcase remarkable abilities, yet they struggle with limitations such as hallucinations, outdated knowledge, opacity, and inexplicable reasoning. To address these challenges, Retrieval-Augmented Generation (RAG) has proven to be a viable solution, leveraging external databases to improve the consistency and coherence of generated content, especially valuable for complex, knowledge-rich tasks, and facilitates continuous improvement by leveraging domain-specific insights. However, RAG is not without its limitations, including a limited context window, irrelevant information, and the high processing overhead for extensive contextual data. In this comprehensive work, we explore the evolution of Contextual Compression paradigms, providing an in-depth examination of the field. We also introduce a state-of-the-art evaluation framework and benchmark. Finally, we outline the current challenges and suggest potential research and development directions, paving the way for future advancements in this area
Paper Type: Long
Research Area: Language Modeling
Research Area Keywords: retrieval-augmented models, applications, robustness, retrieval-augmented generation, NLP in resource-constrained settings, retrieval,
Contribution Types: Surveys
Languages Studied: Multiple languages
Submission Number: 235
Loading