Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View

Jintian Zhang, Xin Xu, Shumin Deng

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Multi-Agent, Large Language Model, Society of Mind, Social Psychology, Human Group Dynamics

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

As Natural Language Processing (NLP) systems are increasingly employed in intricate social environments, a pressing query emerges: \emph{Can these NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models (LLMs)?}
This paper probes the collaboration mechanisms among contemporary NLP systems by melding practical experiments with theoretical insights. We fabricate four unique 'societies' comprised of LLM agents, where each agent is characterized by a specific 'trait' (easy-going or overconfident) and engages in collaboration with a distinct 'thinking pattern' (debate or reflection).
Evaluating these multi-agent societies on three benchmark datasets, we discern that LLM agents navigate tasks by leveraging diverse social behaviors, from active debates to introspective reflections.
Notably, certain collaborative strategies only optimize efficiency (using \textit{fewer} API tokens), but also outshine previous top-tier approaches.
Moreover, our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity or majority rule, mirroring foundational Social Psychology theories.
In conclusion, we integrate insights from Social Psychology to contextualize the collaboration of LLM agents, inspiring further investigations into the collaboration mechanism for LLMs.
We commit to sharing our code and datasets (already submitted in supplementary materials), hoping to catalyze further research in this promising avenue.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2639