Evaluate, Scale, and Credit: A Comprehensive Study on Multi-Agent Collaboration of Large Language Models
Abstract: Large Language Models based Multi-Agent Systems (LLM-MAS) perform well in many domains, but we still lack a clear understanding of the collaboration mechanism among multiple LLM-based agents. This study aims to explore three key issues: (1) Can multi-agent outperform single-agent systems? (2) Is scaling better for multi-agent systems? (3) How to credit agents and find potential effective structures? Specifically, we design five collaboration architectures and evaluate their effectiveness across different LLMs and tasks. Our findings offer significant insights for understanding the collaboration within MAS, building collaboration architectures among agents, and reducing system costs. Furthermore, our conclusion will inspire and provide new perspectives for future studies on LLM-MAS.
Paper Type: Long
Research Area: Generation
Research Area Keywords: interactive and collaborative generation
Contribution Types: NLP engineering experiment, Data analysis
Languages Studied: English
Submission Number: 2648
Loading