Recursive Reasoning as Attractor Landscape Search: Mechanistic Dynamics of the Tiny Recursive Model

Andreas Efstathiou, Aishwarya Balwani

Track: tiny / short paper (up to 5 pages)

Keywords: latent reasoning, recursive reasoning models, mechanistic interpretability, attractor dynamics, test-time scaling

TL;DR: Mechanistic analysis of the Tiny Recursive Model reveals reasoning operates as search through attractor landscapes rather than incremental refinement.

Abstract:

Current reasoning systems predominantly rely on chain-of-thought approaches that generate explicit reasoning tokens, though recent work has begun exploring alternative architectures.
Latent reasoning models such as the Tiny Recursive Model (TRM) recursively refine continuous hidden states rather than producing readable intermediate steps, and have demonstrated competitive performance with large language models while using orders of magnitude fewer parameters.
However, these models remain poorly understood mechanistically -- most evaluations focus on behavioral outcomes rather than the internal dynamics that produce them, limiting our ability to improve their reliability and efficiency.
We therefore conduct a mechanistic analysis of TRM's recursive dynamics on Sudoku-Extreme, revealing three critical insights:
(i) Sparse autoencoder analysis shows models form initial hypotheses early, after which feature activations converge to approximately periodic patterns that primarily stabilize rather than refine solutions;
(ii) Reasoning trajectories in latent principal component space diverge within early recursion steps and descend into local minima determined by initialization, validating similar findings in the Hierarchical Reasoning Model;
(iii) During inference, failed runs plateau at stable high-loss attractors rather than continuing exploration.
Leveraging these insights, we demonstrate that simple inference-time interventions, i.e., adding noise to answer initialization and scaling guess attempts, enable escape from local minima, achieving near-state-of-the-art performance on Sudoku-Extreme using vanilla TRM without retraining.
Our findings suggest recursive reasoning operates as adaptive search through an attractor landscape rather than incremental refinement, opening new avenues for test-time scaling.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 91