Bridging Cross-Chunk Gaps: A Self-Questioning Approach for Long-Context Retrieval-Augmented Generation

ACL ARR 2025 February Submission63 Authors

Abstract: Retrieval-Augmented Generation (RAG) has been broadly adopted to mitigate hallucinations in large language models (LLMs) by grounding their outputs in external documents. However, when dealing with long, coherently structured texts, the standard assumption that each chunk is self-contained often fails—vital context may span multiple segments. This breakdown undermines retrieval reliability and ultimately impairs generation quality. Our empirical findings reveal that in long-document scenarios, as many as 92\% of user queries require cross-chunk semantic dependencies to produce sufficiently supported answers. This observation aligns with cognitive frameworks like the Zeigarnik Effect and Kintsch’s Construction-Integration Model, both emphasizing the need to track incomplete information until a coherent whole is formed. To address these challenges, we propose a Self-Questioning RAG (SqRAG) framework. The core idea is to generate and integrate question–answer pairs that explicitly capture inter-chunk connections, thereby enhancing the retrieval process to account for global context rather than isolated segments. Experimental evaluations demonstrate that our approach not only reduces hallucinations but also improves coherence and factual accuracy across multiple benchmarks, confirming that modeling cross-chunk dependencies is key to robust and context-rich generation.

Paper Type: Long

Research Area: Question Answering

Research Area Keywords: Retrieval Augmented Generation, Long Context Retrieval Augmented Generation, Large Language Models

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data resources, Data analysis

Languages Studied: English, Chinese

Submission Number: 63