Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs

ICLR 2026 Conference Submission19690 Authors

Keywords: large language models, reasoning, knowledge graphs

Abstract: Large language models (LLMs) have demonstrated impressive reasoning abilities yet remain unreliable on knowledge-intensive, multi-hop questions---they miss long-tail facts, hallucinate when uncertain, and their internal knowledge lags behind real-world change. Knowledge graphs (KGs) offer a structured source of relational evidence, but existing KGQA methods face fundamental trade-offs: compiling complete SPARQL queries without knowing available relations proves brittle, retrieving large subgraphs introduces noise, and complex agent frameworks with parallel exploration exponentially expand search spaces. To address these limitations, we propose Search-on-Graph (SoG), a simple yet effective framework that enables LLMs to perform iterative informed graph navigation using a single, carefully designed \textsc{Search} function. Rather than pre-planning paths or retrieving large subgraphs, SoG follows an ``observe, then navigate'' principle: at each step, the LLM examines actual available relations from the current entity before deciding on the next hop. This approaches further adapts seamlessly to different KG schemas and handles high-degree nodes through adaptive filtering. Across six KGQA benchmarks spanning Freebase and Wikidata, SoG achieves state-of-the-art performance using off-the-shelf LLMs without fine-tuning. We demonstrate particularly strong gains on Wikidata benchmarks (+15\% improvement over previous best methods) alongside consistent improvements on Freebase benchmarks.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 19690