Context Learning for Multi-Agent Discussion

Xingyuan Hua, Sheng Yue, Xinyi Li, Yizhe Zhao, Jinrui Zhang, Ju Ren

Keywords: Large Language Models, Context Learning, Multi-agent discussion

TL;DR: We introduce a new context learning method for multi-LLM systems that can continually adjust LLMs' context based on the current state of discussion, enabling agents to effectively collaborate.

Abstract: Multi-Agent Discussion (MAD) has garnered increasing attention very recently, where multiple LLM instances collaboratively solve problems via structured discussion. However, we find that current MAD methods easily suffer from discussion inconsistency—LLMs fail to reach a coherent solution—due to the misalignment between their individual contexts. In this paper, we introduce a multi-LLM context learning method (M2CL) that learns a context generator for each agent, capable of dynamically generating context instructions per discussion round via automatic information organization and refinement. Specifically, inspired by our theoretical insights on the context instruction, M2CL trains the generators to control context coherence and output discrepancies via a carefully crafted self-adaptive mechanism. It enables LLMs to avoid premature convergence on “majority noise” and progressively reach the correct consensus. We evaluate M2CL on challenging tasks, including academic reasoning, embodied tasks, and mobile control. The results show that the performance of M2CL significantly surpasses existing methods by 20\%--50\%, while enjoying favorable transferability and computational efficiency.\footnote{Code is available at \url{https://github.com/HansenHua/M2CL-ICLR26}.}

Primary Area: generative models

Submission Number: 8412