Can RAG Models Know What They Don't Know? Analyzing and Improving Knowledge Boundary Perception

ACL ARR 2026 January Submission5096 Authors

Keywords: Knowledge Boundary Perception, Abstain, Retrieval-augmented Generation, Control Generation

Abstract: Retrieval-Augmented Generation (RAG) provides models with external knowledge to help mitigate hallucinations, but this external knowledge may contain irrelevant, distracting, or conflicting contents. This paper investigates the impact of external knowledge on model's internal perception of knowledge boundaries. We first conduct experiments to compare different detection methods with and without external documents, which reveal that external knowledge impairs models' ability to distinguish between known and unknown information, causing them to treat the unknown as known. Building on this finding, we refine training strategies to enhance the perception of knowledge boundary and propose a knowledge-boundary-based controlled generation framework. This enables models to dynamically determine knowledge reliance and reject unknown questions. Experiments demonstrate that our framework substantially improves generation quality with negligible additional training overhead. Code is submitted with the paper and will be publicly available.

Paper Type: Long

Research Area: Question Answering

Research Area Keywords: interpretability, generalization,open-domain QA

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 5096