Modeling Tool Use in Transformers via Computation Oracles

Keywords: Tools, Agents, Function Calls, Expressivity, Oracles, Computation Theory, Complexity Theory

TL;DR: We propose a new formal framework for analysing the popular setting of allowing agents access to tools/function calls and draw equivalence with test-time depth scaling techniques and prove addition results.

Abstract: Prior literature has mapped the transformer architecture to classical models of computation and especially via circuit complexity, and analyzed how expressive power gets enhanced by adding computational resources like Chain-of-Thought, padding tokens, and depth scaling. Adding tools or function calls to transformer models has shown impressive empirical gains but their theoretical understanding remains under-studied. In this paper, we analyze function calls as $oracles$ in classical complexity theory and provide a formal framework to analyze the expressive gains of augmenting transformer models with function calls and agentic cooperation. We show that (i) fixed-depth transformers with logarithmically many oracle calls can decide $STCON$ via a weak state-transition oracle, (ii) repeated-block (unrolled) universal transformers are equivalent to bounded-bandwidth oracle computation when the oracle is block-realizable, and (iii) access to threshold decision oracles suffices to compute associated optimization objectives via binary search using only $O(\log B(n))$ adaptive queries, with between-query control logic implementable in constant-depth $SMAT$. Together, these results provide a complexity-theoretic framework for understanding tool use as an alternative mechanism for scaling test-time compute while clarifying when tool access can (and cannot) exceed the power of log-depth transformers.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Style Files: I have used the style files.

Submission Number: 121