G1: Teaching LLMs to Reason on Graphs with Reinforcement Learning

Xiaojun Guo, Ang Li, Yifei Wang, Stefanie Jegelka, Yisen Wang

Keywords: Graph, Large Language Models, Reasoning, Reinforcement Learning

Abstract: Despite the success of Large Language Models (LLMs), their performance in graph-related tasks remains limited. Prior approaches, such as pretraining graph foundation models or supervised fine-tuning, struggle due to the lack of large-scale, universally represented graph data. We propose G1, a simple yet effective method that uses Reinforcement Learning (RL) on synthetic graph-theoretic tasks to enhance LLMs' graph reasoning. We introduce Erdos, the largest graph reasoning dataset to date, with 50 diverse tasks, 100k training, and 5k test samples derived from real-world graphs. With RL on Erdos, G1 achieves significant improvements—our 3B model even surpasses Qwen2.5-72B-Instruct (24x larger). The RL-trained models generalize zero-shot to unseen tasks, domains, and graph encodings, including other graph-theoretic benchmarks and real-world node classification and link prediction tasks, without losing general reasoning abilities. Our work demonstrates that RL on synthetic graph tasks efficiently unlocks LLMs' latent graph understanding, offering a scalable path for building powerful graph reasoners.

Submission Number: 96