Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?

Franka Bause, Jonas Niederle, Martin Pawelczyk, Rebekka Burkholz

Keywords: AI agents, LLMs, Multi-Agent Systems, Mixture-of-Experts, Opinion Dynamics, Influence

TL;DR: We cast multi-agent LLM systems as mixture of experts and analyze how agents develop influence during deliberations.

Abstract: The effectiveness of multi-agent LLM deliberation depends not only on the agents' individual predictions, but also on how they communicate and collaborate. We study this mechanism through the lens of Friedkin-Johnsen (FJ) opinion dynamics, a tractable model for analyzing stubbornness, influence, and opinion change in multi-agent systems that captures empirically observed deliberation patterns. We show that the FJ parameters are input dependent, turning multi-agent deliberation into a mixture of experts. This perspective implies that multi-agent systems can outperform single agents and static ensembles when routing reflects agent competence. Since competence is latent in practice, we analyze how influence is established through observable proxies: agents' self-assessed confidence, their perceived confidence, and initial alignment with other agents' views.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 183