More Agents Is All You Need

junyou li, Qin Zhang, Yangbin Yu, QIANG FU, Deheng Ye

Abstract: We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method, termed as Agent Forest, is orthogonal to existing complicated methods to further enhance LLMs, while the degree of enhancement is correlated to the task difficulty. We conduct comprehensive experiments on a wide range of LLM benchmarks to verify the presence of our finding, and to study the properties that can facilitate its occurrence. Our code is publicly available at: https://github.com/MoreAgentsIsAllYouNeed/AgentForest

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=6pNSLmLDHa

Changes Since Last Submission: Based on the reviewers' feedback, we have revised the original manuscript.

Code: https://github.com/MoreAgentsIsAllYouNeed/AgentForest

Assigned Action Editor: ~Karthik_R_Narasimhan1

Submission Number: 2706