Augmenting LLM-based Zero-Shot Retrieval with Listwise Ranking

ACL ARR 2026 January Submission10334 Authors

Keywords: reranking, query augmentation, retrieval

Abstract: Modern IR systems rely on retrieval-reranking pipelines that combine efficient first-stage retrieval with accurate yet expensive second-stage ranking. While distilling ranking knowledge into retrievers makes retrieval effective, existing approaches require supervision and are unsuitable for zero-shot settings. We propose LRaR, a zero-shot retrieval framework that integrates discriminative listwise ranking signals into corpus-wide retrieval using a single LLM call. LRaR jointly performs query augmentation and extracts ranking preferences from a small set of initial candidates, mapping them into a retrieval-compatible representation without training or relevance annotations. Experiments on DL19, DL20, and BEIR show that LRaR consistently improves LLM-based retrieval across diverse LLMs, matching or outperforming reranking over 100 documents using signals from only the top 20, while being substantially more efficient.

Paper Type: Short

Research Area: Information Extraction and Retrieval

Research Area Keywords: passage retrieval

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 10334