EG-RAG: From Passages to Graph Triples for Explainable LLM Factuality

ACL ARR 2026 January Submission10589 Authors

Keywords: LLM, RAG, Graph RAG, Explainability, Fact checking, LLM-as-a-Judge

Abstract:

This work introduces a graph-based retrieval-augmented generation (RAG) approach aimed at validating large language models (LLMs) to enhance their factual accuracy and explainability. While most existing RAG architectures prioritize reducing hallucinations, they often neglect interpretable justifications, leaving explainability underexplored. We argue that hallucination metrics alone are insufficient without interpretable justifications. To this end, we propose an approach that utilizes knowledge graph extraction of queries and claims with graph context matching to provide evidence. The approach includes steps for reasoning related subgraphs using text embedding language models and the all-Steiner tree method, validation, and explanation.

Our method explicitly derives LLM validation and interpretability from the semantic relationships that represent atomic facts. As a step toward explainability, we generate explanations for these relationships and propose explanation-based reasoning using prominent atomic facts from the Knowledge graphs.

Our experiments demonstrate that our proposed explanation-based reasoning improves the factual accuracy of several LLMs, up to 40 percent, on the Fever and Pub Health datasets. Our explainability experiments demonstrate that our method improves judgment, and the digested inference context it generates yields LLM inferences that are more accurate than those generated from textual LLM context. In addition, our method transparently exposes matched entities from the knowledge graph of facts and the user query.

Paper Type: Long

Research Area: Retrieval-Augmented Language Models

Research Area Keywords: Generation, Language Modeling, NLP Applications, Information Extraction

Contribution Types: Model analysis & interpretability, Data resources, Data analysis, Theory

Languages Studied: English

Submission Number: 10589