Amortized Latent Steering: Low-Cost Alternative to Test-Time Optimization

Nathan Egbuna, Saatvik Gaur, Kevin Zhu, Sunishchal Dev, Ashwinee Panda, Maheep Chaudhary

Keywords: efficient reasoning, test time optimization, latent space steering, computational efficiency

TL;DR: ALS achieves 2-5× speedup over iterative test-time optimization by amortizing expensive per-query optimization into a single offline-computed steering vector, matching CoT performance on reasoning tasks.

Abstract: Test-time optimization remains impractical at scale due to prohibitive inference costs---techniques like iterative refinement and multi-step verification can require $10-100\times$ more compute per query than standard decoding. Latent space test-time optimization methods like LatentSeek offer a more direct approach by steering hidden representations, but still demand expensive per-query optimization loops with multiple backward passes. We propose Amortized Latent Steering (ALS), which collapses this iterative optimization into a single offline-computed vector applied at constant cost during inference. ALS computes the mean difference between hidden states from successful versus unsuccessful generations, then uses this direction to calibrate the model’s hidden representations: when decoding drifts away from the success manifold, ALS nudges activations back toward it. Across GSM8K and MATH-$500$ benchmarks, ALS achieves $2-5\times$ speedup over iterative methods while matching or surpassing greedy Chain-of-Thought and Self-Consistency baselines, yielding up to 101% improvement in efficiency–accuracy trade-off. These results show that much of latent optimization's benefit can be captured offline, making sophisticated reasoning techniques viable for production deployment. Code is available at https://github.com/negbuna/ALS.

Submission Number: 208