CORE: Discovering Intrinsic Ranking Preferences in LLMs via Consistent Ego-Correction

ACL ARR 2026 January Submission3271 Authors

Keywords: Listwise Reranking, Large Language Models, Prompt Sensitivity, Information Retrieval, Robustness

Abstract: Large language models are powerful listwise rerankers, yes their performance remains highly sensitive to prompt variations, undermining their reliability for real-world applications. To address this, we propose CORE, a new fine-tuning framework that mitigates this instability by learning a model's intrinsic, prompt-invariant ranking preferences. CORE integrates two complementary mechanisms: a guidance strategy adapted from Classifier-Free Guidance to calibrate the generative process against stylistic variations, and a consistency loss based on differentiable Kendall's Tau to regularize the model's internal ordinal judgments. On standard TREC Deep Learning and BEIR benchmarks, CORE establishes new state-of-the-art ranking performance. Crucially, CORE demonstrates superior robustness, reducing performance variance across diverse prompts by over 80\% compared to standard fine-tuning. Our work presents a principled and effective method for building powerful and trustworthy LLM-based reranking systems.

Paper Type: Long

Research Area: Information Extraction and Retrieval

Research Area Keywords: re-ranking, prompting, robustness, fine-tuning

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 3271