SPARKLE: A Structured and Plug-and-play Agentic Retrieval Policy for Adaptive RAG Models

ACL ARR 2026 January Submission6638 Authors

Keywords: Retrieval-Augmented Generation, Adaptive RAG, Proxy Model, Knowledge Graph, LLM

Abstract: Adaptive retrieval-augmented generation (RAG) models offer an effective approach for integrating external knowledge. However, existing methods either rely on frozen large language models (LLMs) without explicit supervision or require costly LLM finetuning. Therefore, we propose SPARKLE, a structured and plug-and-play agentic retrieval policy where an additional proxy model is introduced to control the retrieval process. The proxy model leverages knowledge graph-based reasoning to make retrieval decisions in a structured manner, while operating independently of the retriever and the LLM. This plug-and-play design allows SPARKLE to generalise across different retrievers and LLMs. SPARKLE is optimised via reinforcement learning (RL), treating the retriever and the LLM as part of the environment. To enable more effective exploration during RL training, we further introduce a binary tree-structured rollout strategy. Experiments on three in-domain and four out-of-domain QA benchmarks show that SPARKLE outperforms state-of-the-art adaptive RAG baselines, achieving average improvements of 9.17% and 2.85%, respectively.

Paper Type: Long

Research Area: Retrieval-Augmented Language Models

Research Area Keywords: Generation, Language Modeling, Question Answering; LLM

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 6638