RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing

ACL ARR 2026 January Submission8119 Authors

Keywords: Adaptive Retrieval, RAG Routing, Query-Corpus Compatibility

Abstract: Retrieval-Augmented Generation (RAG) has become a core paradigm for grounding large language models with external knowledge. Despite extensive efforts exploring diverse retrieval strategies, existing studies predominantly focus on query-side complexity or isolated method improvements, lacking a systematic understanding of how RAG paradigms behave across different query–corpus contexts and effectiveness–efficiency trade-offs. In this work, we introduce RAGRouter-Bench, the first dataset and benchmark designed for adaptive RAG routing. RAGRouter-Bench revisits retrieval from a query–corpus compatibility perspective and standardizes five representative RAG paradigms for systematic evaluation across 7,727 queries and 21,460 documents spanning diverse domains. The benchmark incorporates three canonical query types together with fine-grained semantic and structural corpus indicators, as well as a unified protocol that jointly evaluates generation quality and resource consumption. Experiments with DeepSeek-V3 and LLaMA-3.1-8B demonstrate that no single RAG paradigm is universally optimal, that paradigm applicability is strongly shaped by query–corpus interactions, and that increased retrieval complexity does not necessarily yield better effectiveness–efficiency trade-offs. These findings underscore the necessity of routing-aware evaluation and establish a foundation for adaptive, interpretable, and generalizable next-generation RAG systems. The code and dataset are available at: \url{https://anonymous.4open.science/r/RAGRouter-Bench/}.

Paper Type: Long

Research Area: Retrieval-Augmented Language Models

Research Area Keywords: Retrieval-Augmented Generation

Contribution Types: Data resources, Data analysis

Languages Studied: English

Submission Number: 8119