From Retrieval to Reranking: Evaluating LLM Strategies for Long-Form Cases

ACL ARR 2026 January Submission10053 Authors

Keywords: case retrieval, reranking, large language models (LLMs), long-context retrieval

Abstract: Case retrieval is a critical component of case-based reasoning in domains such as law and medicine, where decisions are informed by prior cases. The task is particularly challenging because both queries and candidate cases are often extremely long with relevant evidence sparsely distributed across lengthy texts. We systematically study two aspects of long case retrieval. First, we compare full-document embeddings with LLM-generated summaries, finding that summaries improve retrieval performance for weaker methods such as BM25 while full-context representations are more effective for embeddings produced by strong LLMs such as Qwen3. Second, we examine reranking strategies, contrasting retrieval heads that capture token-level evidence with LLM-based rerankers that perform higher-level reasoning. Experiments show complementary strengths: retrieval heads excel with strong query-document overlap, while LLM-based rerankers perform better when complex reasoning is needed. Our findings provide guidance for designing retrieval systems that balance context coverage, token-level similarity, and reasoning for long-form cases.

Paper Type: Short

Research Area: Question Answering

Research Area Keywords: open-domain QA

Contribution Types: Model analysis & interpretability

Languages Studied: english

Submission Number: 10053