Hidden Markov Modeling of Reasoning Dynamics in Large Language Models

ICLR 2026 Conference Submission21865 Authors

Keywords: Reasoning dynamics, Hidden Markov Model, Transition analysis, Large language models, Interpretability.

Abstract: Reasoning in language models involves both explicit steps in the generated text and implicit structural shifts in hidden states, yet their joint dynamics remain largely underexplored. We propose a Hierarchical Hidden Markov Model (HHMM) that captures these two dimensions: semantic roles and latent depth regimes. This framework models how reasoning evolves through semantic stages and how the depth of computation shifts across the network. By linking what function a step serves to where it arises in the network, our approach provides a unified lens for both understanding reasoning dynamics and offering insights into steering strategies. Our analysis reveals consistent patterns: successful reasoning trajectories follow stable semantic paths and align with well-formed structural anchors, whereas failures are characterized by hesitation loops and unstable depth transitions. We further validate our findings by applying step-aware intervention: we derive steering vectors from the transition matrices that encourage trajectories to follow the paths associated with correct reasoning. Across multiple open-source reasoning models, these targeted nudges consistently convert failing runs into correct ones without increasing output length.

Primary Area: interpretability and explainable AI

Submission Number: 21865