From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs

ACL ARR 2026 January Submission663 Authors

Keywords: Multihop QA; LLM; Knowledge Graph;

Abstract:

Large Language Models (LLMs), despite their success in question answering, exhibit limitations in complex multi-hop question answering (MQA) tasks that necessitate non-linear, structured reasoning.
This limitation stems from their inability to adequately capture deep conceptual relationships between entities. To overcome this challenge, we present **ORACLE** (**O**ntology-driven **R**easoning **A**nd **C**hain for **L**ogical **E**lucidation), a training-free framework that combines LLMs' generative capabilities with the structural benefits of knowledge graphs.
Our approach operates through three stages: (1) dynamic construction of question-specific knowledge ontologies using LLMs, (2) transformation of these ontologies into First-Order Logic (FOL) reasoning chains, and (3) systematic decomposition of the original query into logically coherent sub-questions.
Extensive experiments across a diverse set of models and standard MQA benchmarks demonstrate that our framework achieves competitive performance while producing more interpretable reasoning chains.

Paper Type: Long

Research Area: Question Answering

Research Area Keywords: multihop QA; reasoning;

Contribution Types: Approaches to low-resource settings, Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 663