FRAG: Filtering Noise Using Snippet-Level Query Relevance

ICLR 2026 Conference Submission12973 Authors

Keywords: LLM, RAG, Snippet-level Query Relevance

Abstract: Retrieval-Augmented Generation (RAG) augments large language models (LLMs) with external retrievals. Typically, expanding the retrieval window can improve RAG performance by retrieving more relevant content. However, it risks increased noise, which distracts the model’s attention and degrades accuracy. To mitigate this, we propose Fine-Grained RAG (FRAG), which identifies key snippets from query and extracts relevant information while filtering noise from retrievals using snippet-level query relevance. Yet, a new challenge arises in addressing complex RAG queries, which require knowledge pieces with implicit multi-hop logical relationships. Failure to identify these relationships may lead to loss of inference-based knowledge during filtering, degrading performance. To address this, we propose Self-Recognition, which extracts inference-based knowledge by leveraging historically extracted knowledge as contextual references. While FRAG notably improves performance, it incurs high computational cost. To alleviate this, we present FRAG-ip, a fine-tuned framework which markedly accelerates FRAG by an order of magnitude. Extensive experiments show that FRAG significantly boosts RAG, yielding average accuracy gains of 4.94%/13.44% on simple/complex tasks.

Primary Area: generative models

Submission Number: 12973