SAGE: Strategy-Adaptive Generation Engine for Query Rewriting

ACL ARR 2025 July Submission250 Authors

Abstract:

Query rewriting is pivotal for enhancing dense retrieval, yet current methods demand large-scale supervised data or suffer from inefficient reinforcement learning (RL) exploration.
In this work, we first establish that guiding Large Language Models (LLMs) with a concise set of expert-crafted strategies substantially improves retrieval effectiveness on challenging benchmarks, including HotpotQA, FEVER, NFCorpus, and SciFact.
Building on this insight, we introduce the Strategy-Adaptive Generation Engine (SAGE), which operationalizes these strategies in an RL framework.
SAGE introduces two novel reward shaping mechanisms- Strategic Credit Shaping (SCS) and Contrastive Reward Shaping (CRS)-to deliver more informative learning signals.
This strategy-guided approach not only achieves new state-of-the-art NDCG@10 results, but also uncovers a compelling emergent behavior:
the agent learns to select optimal strategies, reduces unnecessary exploration, and generates concise rewrites, lowering inference cost without sacrificing performance.
Our findings demonstrate that strategy-guided RL, enhanced with nuanced reward shaping, offers a scalable, efficient, and more interpretable paradigm for developing the next generation of robust information retrieval systems.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: Query Rewrite, Reinforcement Learning, Large Language Model

Contribution Types: NLP engineering experiment, Reproduction study, Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 250