GRAPE: Let GPRO Supervise Query Rewriting by Ranking for Retrieval

ICLR 2026 Conference Submission17830 Authors

Keywords: CLIP, Large Language Models (LLMs), Retrieval, Query Rewriting

Abstract: The CLIP model has become a cornerstone of large-scale retrieval systems by aligning text and image data in a unified embedding space. Despite its simplicity and efficiency, CLIP struggles when applied to tasks whose input distributions diverge from its training corpus, such as queries with multilingual, long-form, or multimodal differences. To avoid costly retraining, existing methods mainly adopt query-rewriting strategies with large language models (LLMs), aiming to mitigate distribution gaps at the query level. However, due to the lack of supervision signals, LLMs fail to generate the optimal one that fits the training distribution. We address this challenge with GRAPE (\textbf{G}rouped \textbf{R}anking-\textbf{A}ware \textbf{P}olicy Optimization \textbf{E}nhancement), a plug-and-play enhancement approach that incorporates ranking signals into retrieval-guided query rewriting with LLMs. Intuitively, GRAPE proposes to leverage GRPO to bridge distributional differences—including length, multilingual, and modality shifts—by transforming queries into forms better aligned with the retriever’s training distribution. However, our preliminary experiment finds that naively finetuning LLM with similarity scores can lead to score inflation, where nearly all candidates are assigned unexpectedly high scores regardless of their true relevance. To address score inflation, we propose a corpus-relative ranking-based reward, which explicitly aligns optimization with ranking metrics while suppressing spurious score inflation. Extensive experiments demonstrate that GRAPE consistently improves retrieval performance under distributional shifts—including multilingual differences (Flickr30k-CN, CVLUE, XM3600), length differences (Wikipedia), and multimodal differences (CIRR)—achieving an average improvement of 4.9\% in Recall@10.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 17830