Hetero-Designer: Automated Design of Multi-Agent Systems with Heterogeneous LLMs

ACL ARR 2026 January Submission8598 Authors

Keywords: Autonomous agents, LLM agents, multi-agent systems, agent communication

Abstract:

LLM-based Multi-agent systems (MAS) have shown strong capabilities across a wide range of domains.
Their success largely hinges on the collaboration topology design, which has emerged as a central research focus in the automated MAS design.
However, existing approaches are fundamentally constrained by their reliance on homogeneous LLMs, which significantly limits overall system intelligence.
In response to this limitation, we for the first time propose the concept of **Automated Design of Heterogeneous-LLMs-based MAS (ADHM)**.
ADHM sheds light on a promising avenue for advancing collective intelligence, which focuses on the automated design of cost-efficient MAS composed of diverse LLMs
and roles to suit various queries.
Toward this challenging goal, we propose **Hetero-Designer**, a novel pipeline that efficiently encodes intricate dependencies among queries, LLMs and roles through a novel Binary-Star Transformer and constructs Hetero-MAS in an autoregressive graph generation process. Extensive experiments demonstrate that **Hetero-Designer** is: (1) high-performing on various benchmarks, (2) economical in reducing overhead, (3) extensible to unseen LLMs and roles.

Paper Type: Long

Research Area: AI/LLM Agents

Research Area Keywords: Autonomous agents,LLM agents,multi-agent systems,agent communication

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 8598