Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning

ACL ARR 2025 May Submission4823 Authors

Abstract:

Existing document reranking methods based on large language models (LLMs) typically rely on prompting or fine-tuning LLMs to order or label candidate documents according to their relevance to a query. In this paper, we introduce Rank-R1, an LLM-based reranker that performs reasoning over both the user query and candidate documents before performing the ranking task, by training with reinforcement learning along with only a small set of relevance labels (without any reasoning supervision) to enhance the reasoning ability of LLM-based rerankers. Our hypothesis is that adding reasoning capabilities to the rerankers can improve their relevance assessment and ranking capabilities. Our experiments on the TREC DL and BRIGHT datasets show that Rank-R1 is highly effective, especially for complex queries.
In particular, we find that Rank-R1 achieves effectiveness on in-domain datasets at par with that of supervised fine-tuning methods.
In addition, the model largely outperforms zero-shot and supervised fine-tuning when applied to out-of-domain datasets featuring complex queries, especially when a 14B-size model is used.

Paper Type: Short

Research Area: Information Retrieval and Text Mining

Research Area Keywords: re-ranking

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 4823