Towards Efficient and Scalable Multi-agent Reasoning via Bayesian Nash Equilibrium
Keywords: Large Language Models, Reasoning, Multiagent Reasoning
Abstract: Large Language Models (LLMs) have achieved significant success in tasks such as natural language understanding, generation, and reasoning, driving advancements in machine translation, summarization, and question-answering systems. Particularly in multi-step reasoning tasks, LLMs demonstrate robust logical reasoning capabilities. To further enhance the reasoning abilities of LLMs, researchers have developed methods that generate intermediate reasoning steps, thereby improving performance in solving complex problems. However, these approaches primarily focus on single LLMs, limiting the diversity and creativity of the reasoning process. Inspired by psychological studies, there is a growing interest in exploring multi-LLM frameworks to boost model performance through collaborative reasoning. Existing multi-agent debate frameworks can enhance answer accuracy through dialogue and argumentation but suffer from high computational costs and lack theoretical guarantees for convergence. To address these challenges, we propose a novel method—BNE-Q. This method integrates belief networks and a central network within the Decentralized Partially Observable Markov Decision Process (DEC-POMDP) framework to achieve a Bayesian Nash Equilibrium (BNE). During the inference phase, the central LLM provides step-by-step strategies and format guidelines, while execution LLMs independently generate answers based on these guidelines. The central LLM then consolidates the answers to form a commitment. In the optimization phase, by calculating the cosine similarity between the answers and the commitment, we optimize the execution LLMs' belief networks to achieve stable convergence of the multi-agent system. Our method not only reduces the computational overhead compared to traditional multi-agent debate methods but also ensures convergence through theoretical analysis. Experimental results demonstrate that BNE-Q performs exceptionally well across six benchmark tests, including complex reasoning and planning tasks, validating its theoretical robustness and practical effectiveness. Our work provides a new approach for developing multi-LLM collaborative reasoning frameworks, significantly enhancing reasoning capabilities in large-scale multi-agent environments.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 14101
Loading