Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives

Elliot Meyerson, Xin Qiu

TL;DR: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives

Abstract: Decomposing hard problems into subproblems often makes them easier and more efficient to solve. With the high cost of running LLMs at scale, there is an increasing effort to decompose systems into sets of LLM-based agents, each of whom can be delegated sub-tasks. However, this decomposition (even when automated) is often intuitive, e.g., based on how a human might assign roles to members of a human team. How close are these role decompositions to optimal? This position paper argues that asymptotic analysis with LLM primitives is needed to reason about the efficiency of such problem decompositions, and that insights from such analysis will unlock opportunities for scaling such systems. By treating the LLM forward pass as the atomic unit of computational cost, one can separate out the (often opaque) inner workings of a particular LLM from the inherent efficiency of how a set of LLMs are orchestrated to solve hard problems. In other words, if we want to scale the deployment of LLMs to the limit, instead of anthropomorphizing LLMs, asymptotic analysis with LLM primitives should be used to reason about and develop more powerful decompositions of large problems into LLM agents.

Lay Summary:

It has become popular to use large language models to build "agents". Each agent is a program with a specific role to play in the world. To solve complex problems, it is natural to build a system with multiple agents, each assigned a complementary role. To date, such agent-based systems have been organized in an intuitive or ad-hoc manner, for example, assigning roles to agents in the same way we would assign jobs to humans in an organization. How close are such agent organizations to optimal? It turns out they are not close at all!

This paper proposes a way to analyze the scalability of such systems, grounded in the classical computer science tool of algorithm analysis. Through a series of examples, we use this approach to show that an intuitive agent-based system can be drastically improved by more carefully considering the algorithmic structure of the problem. This analysis tool will thus help people build agent-based systems that are far more efficient and scalable, and the organization of such future systems will not be humanlike, but uncompromisingly machinelike.

Verify Author Names: My co-authors have confirmed that their names are spelled correctly both on OpenReview and in the camera-ready PDF. (If needed, please update ‘Preferred Name’ in OpenReview to match the PDF.)

No Additional Revisions: I understand that after the May 29 deadline, the camera-ready submission cannot be revised before the conference. I have verified with all authors that they approve of this version.

Pdf Appendices: My camera-ready PDF file contains both the main text (not exceeding the page limits) and all appendices that I wish to include. I understand that any other supplementary material (e.g., separate files previously uploaded to OpenReview) will not be visible in the PMLR proceedings.

Latest Style File: I have compiled the camera ready paper with the latest ICML2025 style files <https://media.icml.cc/Conferences/ICML2025/Styles/icml2025.zip> and the compiled PDF includes an unnumbered Impact Statement section.

Paper Verification Code: OTU2O

Primary Area: Research Priorities, Methodology, and Evaluation

Keywords: Large language models, Agents, Analysis of Algorithms, Decomposition, Anthropomorphism, LLM Primitives, Asymptotic Analysis

Submission Number: 315