FusePRAG: A Fuse-then-Project Paradigm for Parametric Retrieval-Augmented Generation

ACL ARR 2026 January Submission3353 Authors

Keywords: Retrieval Augmented Generation, Hypernetwork

Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) but suffers from high inference latency and noise sensitivity. While Parametric RAG (PRAG) addresses these issues by encoding retrieved knowledge into adapter parameters, existing methods adhere to a "Project-then-Fuse" paradigm, processing documents in isolation. This strategy incurs linear costs and severs cross-document dependencies essential for reasoning. To address this, we propose FusePRAG, a novel framework shifting the paradigm to "Fuse-then-Project". By employing a query-guided fusion module to synthesize semantic logic before projection, FusePRAG generates adapter parameters in a single pass, achieving constant complexity during the projection phase. This architecture constructs a holistic representation of reasoning chains, naturally complementing the fine-grained details of explicit context. Experiments on four benchmarks demonstrate that FusePRAG exhibits robust generalization and yields substantial synergy with standard RAG, achieving superior performance in the hybrid setting.

Paper Type: Long

Research Area: Retrieval-Augmented Language Models

Research Area Keywords: retrieval-augmented generation, reasoning

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 3353