Strategic Insights: Evaluating Large Language Models' Decision-Making in Multi-Player Game-Theoretic Environments

Keywords: Game Theory, Large Language Model, Multi-Agent System, Decision-Making, Strategy, Behavior, Strategic Rationality Score

Abstract: Large Language Models (LLMs) excel in language tasks but their strategic decision-making in interactive, multi-agent scenarios—critical for applications like negotiation systems or social simulations—remains understudied. This paper examines twelve anonymized LLMs in six multi-player game theory scenarios, encompassing cooperative, betraying, and sequential categories, with ten agents per instance across repeated rounds and multiple runs. We propose the Strategic Rationality Score (SRS), a novel composite metric normalizing deviations from Nash equilibria across games, enabling quantitative benchmarking of LLM rationality. Our findings reveal inconsistent equilibrium-seeking behavior, weak correlations with architectural features like parameter size, and minimal adaptation over interactions, suggesting inherent limitations in opponent modeling and long-term reasoning. These results contrast with expectations from scaling laws and highlight biases toward short-term gains. Contributions include SRS for cross-game evaluation, large-scale multi-player simulations (360 instances), and linkages to LLM traits, advancing AI behavioral analysis for safer multi-agent deployments. Data and code are available as *Supplementary Material* (attachment) to this submission, as well as at: https://anonymous.4open.science/r/Agents4Science_2025_LLM_Game_Theory-PPPP.

Submission Number: 61