Go to ACL ARR 2025 February homepageUnderstanding the Why: Leveraging Theme-Based Knowledge Graphs for Enhanced Causal Reasoning
Understanding causality to answer why and how questions is a formidable challenge for Large Language Models (LLMs), particularly in specialized fields requiring extensive domain knowledge. Although advances in retrieval Augmented Generation (RAG) provide LLMs with access to information beyond their training scope, RAG models struggle to infer implicit causal links and build causal narratives, resulting in incomplete and extremely verbose explanations.
We introduce KG-RAG, a novel framework that integrates theme-based knowledge graphs (KGs) with RAG for enhanced causal inference. KG-RAG leverages GPT-4o to extract explicit and implicit $\langle \mathit{cause, relation, effect}\rangle$ triples from domain-specific corpora. These triples are structured into a directed acyclic graph (DAG) to enable multi-hop causal reasoning.
KG-RAG is evaluated on two corpora: Bitcoin price fluctuations, where financial narratives demand high granularity causal inference, and Gaucher disease (GD), a well-researched medical condition with known causal relations. The results show that KG-RAG outperforms GPT-4o with RAG and GPT-4o baselines, achieving higher readability, chain similarity, and conciseness scores. We also evaluated the results using an LLM-as-a-Judge experiment, combining the expertise of 4 state-of-the-art LLMs (GPT-4, GPT-4o, LLaMA 3.1-8B-Instruct, and Mistral-7B-Instruct). The results showed that the performance of the KG-RAG was superior compared to the baselines for both corpora.
Our findings demonstrate the power of integrating structured knowledge graphs into the RAG process to improve causal reasoning in LLMs, paving the way for more interpretable and reliable AI-driven decision making.