From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs

ACL ARR 2026 January Submission6361 Authors

Keywords: retrieval-augmented generation, conflict-aware reasoning, trustworthy RAG systems, evidence adjudication, interpretable reasoning traces, grounded question answering, QLoRA, citation grounding, refusal and abstention, chain-of-thought, temporal and subjective conflicts, evaluation of grounding and correctness, LLM-as-a-judge evaluation, supervised fine-tuning, behavioural adherence

Abstract: Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external evidence, but fails when retrieved sources conflict or contain outdated or subjective information. Prior work address these issues independently but lack unified reasoning supervision. We propose a reasoning-trace-augmented RAG framework that adds structured, interpretable reasoning across three stages: (1) document-level adjudication, (2) conflict analysis, and (3) grounded synthesis, producing citation-linked answers or justified refusals.A Conflict-Aware Trust-Score (CATS) pipeline is introduced which evaluates groundedness, factual correctness, refusal accuracy, and conflict-behavior alignment using an LLM-as-a-Judge. Our 539-query reasoning dataset and evaluation pipeline establishes a foundation for conflict-aware, interpretable RAG systems. Experimental results demonstrate substantial gains over baselines, most notably with Qwen, where Supervised Fine-Tuning (SFT) improved End-to-End answer correctness from 0.069 to 0.883 and behavioral adherence from 0.074 to 0.722.

Paper Type: Long

Research Area: Retrieval-Augmented Language Models

Research Area Keywords: retrieval augmented generation, open domain QA, reasoning, interpretability, robustness, fact checking, misinformation detection, evaluation methodologies, metrics, benchmarking, NLP datasets, reproducibility, fine tuning, chain of thought

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Publicly available software and/or pre-trained models, Data resources, Data analysis

Languages Studied: English (primarily)

Submission Number: 6361