Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning

Martina G. Vilas, Safoora Yousefi, Besmira Nushi, Eric Horvitz, Vidhisha Balachandran

Keywords: Test Time Scaling, Reasoning, Interpretability, Representational Analysis

Abstract:

Reasoning models improve their problem-solving ability through inference-time scaling, allocating more compute via longer and multiple token budgets.
Identifying which reasoning traces are likely to succeed remains a key opportunity: reliably predicting productive thinking paths can substantially reduce wasted computation and improve overall efficiency.
We introduce _Latent-Trajectory_ signals that characterize the temporal evolution of a model's internal representations during the generation of intermediate reasoning tokens.
By analyzing both the extent and temporal course of latent representational change, as well as its alignment with the final state, we show that these signals are strong predictors of solution accuracy, outperforming conventional output-based confidence measures.
We use latent-trajectory signals to guide answer selection across multiple sampled generations, demonstrating that they make test-time scaling more effective and efficient, reducing token usage by up to 70% while preserving and even improving accuracy on average in comparison with majority voting.
Finally, we show that these signals often emerge early in the reasoning trace, which enables early selection and allocation of compute to the most promising answer candidates during generation.
Our findings contribute not only practical strategies for inference-time efficiency, but also a deeper interpretability perspective on how reasoning processes are represented and differentiated in latent space.

Primary Area: interpretability and explainable AI

Submission Number: 14110