InstUPR : Instruction-based Unsupervised Passage Reranking with Large Language Models

Anonymous

Abstract:

This paper introduces InstUPR, a novel unsupervised passage reranking method based on large language models (LLMs).
Different from existing approaches that rely on extensive training with query-document pairs or retrieval-specific instructions, our method leverages the instruction-following capabilities of instruction-tuned LLMs for passage reranking without any additional fine-tuning.
To achieve this, we introduce a soft score aggregation technique and employ pairwise reranking for unsupervised passage reranking.
Experiments on the BEIR benchmark demonstrate that InstUPR outperforms unsupervised baselines as well as an instruction-tuned reranker, highlighting its effectiveness and superiority.

Paper Type: short

Research Area: Information Retrieval and Text Mining

Contribution Types: NLP engineering experiment, Approaches to low-resource settings

Languages Studied: English

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.