AtomGraph: Reasoning Isn't Linear, Why Should Verification Be?

Aryan Karmore

Track: long paper (up to 10 pages)

Keywords: Chain-of-Thought (CoT), Mathematical Reasoning, Graph Neural Network (GNN), Attention Mechanism

Abstract:

Verifying the correctness and consistency of multi-step reasoning chains generated by large language models to solve reasoning problems remains a challenge.
Current verification strategies treat reasoning as strictly linear chains(Chain-of-Thought) or rigid trees (Tree-of-Thought). These approaches fail to capture complex dependencies in mathematical reasoning. AtomGraph is a graph neural network approach that represents reasoning chains as directed acyclic graphs where nodes act as atomic reasoning steps and edges encode logical dependencies. AtomGraph also combines semantic embeddings with structural features then applies multi head graph attention to learn the dependencies which matter the most.
On the GSM8k dataset, AtomGraph achieves 75.9\% F1 score outperforming CoT, ToT baselines by 386\%. Further analysis reveals that AtomGraph learns to assign significantly higher attention to computation nodes than reasoning nodes ($\mu = 0.70$ vs $\mu = 0.478$ ). Skip connections receive higher attention and demonstrate that reasoning verification benefits from explicit modeling of multi-hop dependencies rather than treating reasoning as linear.

Presenter: ~Aryan_Karmore1

Format: Maybe: the presenting author will attend in person, contingent on other factors that still need to be determined (e.g., visa, funding).

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding would significantly impact their ability to attend the workshop in person.

Submission Number: 7