Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains

ICLR 2026 Conference Submission22235 Authors

Keywords: Retrieval-Augmented Generation (RAG), Explainable AI, Interpretable AI, Safe AI, Legal NLP, Scientific NLP, Financial NLP

Abstract: In sensitive domains, Retrieval-Augmented Generation (RAG) must be interpretable and robust because errors here don't just mislead; they invite lawsuits, undermine scholarly credibility, and breach compliance. Stakeholders require traceable evidence, clear rationale for why specific evidence were selected, and safeguards against poisoned or misleading content. Yet current RAG pipelines use similarity-based retrieval with arbitrary top-k cutoffs, offering no explanation for their selections, and remain vulnerable to data poisoning attacks. We propose METEORA, which replaces these existing drawbacks in RAG pipelines with rationale-driven selection; explicit reasoning that simultaneously guides evidence choice, explains decisions, and remains robust to RAG poisoning. METEORA operates in three stages. First, a general-purpose Large Language Model (LLM) is preference-tuned to generate rationales conditioned on the input query using direct preference optimization. Second, these rationales guide the **Evidence Chunk Selection Engine**, which employs a two-step process: (Step 1) pairing individual rationales with retrieved evidence for query-specific relevance and applying elbow detection (identifying sharp drops in similarity scores) to determine an adaptive cutoff point that eliminates the need for top-k heuristics, thereby acquiring dataset-specific relevance, and (Step 2) optionally performing context expansion by adding neighboring evidence. Lastly, the rationales are used by a **Verifier LLM** to detect and filter poisoned or misleading evidence for safe generation. The framework provides explainable and interpretable evidence flow by using rationales consistently across both selection and verification. Our evaluation across six datasets shows METEORA delivers breakthrough performance on three fronts: (i) it achieves **13.41%** higher recall, and its METEORA w/o Expansion variant achieves **21.05%** higher precision than the best-performing baseline; (ii) it reduces the amount of evidence required to reach comparable recall by **80%**, which directly improves downstream answer generation accuracy by **33.34%**; and (iii) it strengthens adversarial defense, increasing F1 **from 0.10 to 0.44** and making RAG systems more resilient to poisoning attacks. Code available at: https://anonymous.4open.science/r/METEORA-DC46/README.md

Primary Area: interpretability and explainable AI

Submission Number: 22235