Follow the Path: Reasoning over Knowledge Graph Paths to Improve LLM Factuality

Mike Zhang, Johannes Bjerva, Russa Biswas

Keywords: large language models, reasoning, factuality, knowledge graphs

TL;DR: This paper demonstrates that fine-tuning Large Language Models on reasoning traces grounded in knowledge graph paths improves their factual accuracy and performance on complex question-answering tasks.

Abstract: We introduce **fs1**, a simple yet effective method that improves the factuality of reasoning traces by sourcing them from large reasoning models (e.g., DeepSeek-R1) and grounding them by conditioning on knowledge graph (KG) paths. We fine-tune eight instruction-tuned Large Language Models (LLMs) on 3.9K factually grounded reasoning traces and rigorously evaluate them on six complex open-domain question-answering (QA) benchmarks encompassing 23.9K questions. Our results demonstrate that our **fs1**-tuned model (32B parameters) consistently outperforms instruction-tuned counterparts with parallel sampling by 6-14 absolute points (pass@$16$). Our detailed analysis shows that \dataset{} considerably improves model performance over more complex questions (requiring 3 or more hops on KG paths) and numerical answer types compared to the baselines. Furthermore, in single-pass inference, we notice that smaller LLMs show the most improvements. While prior works demonstrate the effectiveness of reasoning traces primarily in the STEM domains, our work shows strong evidence that anchoring reasoning to factual KG paths is a critical step in transforming LLMs for reliable knowledge-intensive tasks.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 11827