The Deterministic Horizon: When Extended Reasoning Fails and Tool Delegation Becomes Necessary

Dongxin Guo, Jikun Wu, SM Yiu

Abstract: Extended chain-of-thought reasoning can degrade performance on deterministic state-tracking tasks---not due to preference biases, but fundamental information-theoretic limits in decoder-only transformers. We establish: (1) an Attention Bottleneck Theorem with matching lower bound, proving state-tracking capacity scales as $O(H \cdot \log(L/H) \cdot \sqrt{d_h})$; (2) a context-dependent error model yielding super-exponential accuracy decay; (3) the State-Space Jaccard metric distinguishing capability from preference failures; (4) a Deterministic Horizon $d^* \in [19, 31]$ beyond which tool delegation becomes necessary. Across 12 models and 8 task domains---including SWE-Bench, WebArena, and SQL-Multi---tool-integrated reasoning achieves 86--94\% accuracy versus 24--42\% for neural chain-of-thought. Fine-tuning on optimal-length traces yields $<$5\% improvement, confirming an architectural ceiling. High cross-model correlation ($r = 0.81$--$0.91$) demonstrates these failures are architectural, not training-specific. Our results provide principled guidance for when pure neural reasoning should yield to hybrid approaches in agentic systems.