A Heterogeneous Graph Optimization Perspective for Multi-Agent System Workflows

Bowen Guan, Runze Wang, Yanming Shen

Keywords: LLM Agent; Multi-Agent System

TL;DR: This paper proposes a heterogeneous graph optimization approach for llm-based multi-agent system workflows.

Abstract: Large Language Model (LLM)-based multi-agent systems (MAS) have shown potential in solving complex tasks across a wide range of domains. However, designing effective MAS workflows remains a significant challenge. Manually crafted workflows are difficult to scale and adapt. Automated workflow optimization techniques usually depend heavily on the planning capability of meta-agent, cannot fully utilize the historical context, and neglect the dynamic interactions between agents and tools. To address these limitations, we propose Heterogeneous Graph-based workFlow optimization (HeGFlow), which models agents, tools, and reasoning steps as interconnected graph components, transforming the design of MAS workflow into a heterogeneous graph adjacency matrix optimization problem. To efficiently explore the vast search space, HeGFlow introduces a two-stage matrix training process guided by a subgraph sampling strategy. Extensive experiments across six complex domains show that HeGFlow enables smaller LLMs to match or even surpass the performance of much larger models. Furthermore, HeGFlow consistently outperforms existing manually and automated workflow approaches on four widely-used benchmarks, establishing a new paradigm for scalable and effective MAS workflow generation.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 10789