Dynamic Retrieval AugmentedGeneration Based on The Knowledge-Aware of Large Language Models

Yongliang Wu, Kang Wang, Zhiqiang Wang

Keywords: Retrieval-Augmented Generation, Semantic Drift, Multigranularity Semantic, Prompt Engineering

TL;DR: We propose a dynamic knowledge-aware framework built on a three-stage optimisation strategy for language model Semantic Drift.

Abstract: Retrieval-augmented generation (RAG) has proven effective in enhancing open-domain question answering by supplementing language models with external knowledge. However, current approaches often rely heavily on the model’s internal confidence scores to decide whether retrieval is necessary. This overreliance, coupled with the tendency of language models to show overconfidence, results in excessive and sometimes redundant retrieval operations. Moreover, conventional RAG workflows commonly incorporate coarse-grained retrieved documents directly into the generation process, introducing noise and semantic drift that can compromise answer quality. To address these limitations, we propose DynaRAG, a dynamic knowledge-aware framework built on a three-stage optimisation strategy. First, a hybrid question-knowledge similarity space and a lightweight threshold prediction network are constructed that learns query-adaptive decision boundaries to control retrieval triggering more precisely. Second, we generate multigranularity semantic variants to perform targeted retrieval and rank documents using a newly introduced knowledge importance scoring mechanism, thus improving the relevance and specificity of the retrieved content. Third, a prompt-guided large language model synthesises the final answer based on the refined input of selected knowledge. Extensive experiments demonstrate that DynaRAG achieves average improvements of approximately 11% in EM and 14% in F1 on six representative QA benchmarks. Evaluated against a diverse suite of retrieval-augmented baselines, DynaRAG consistently improves accuracy and efficiency, underscoring its robustness and adaptability in knowledge-intensive tasks.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 5058