GTAlign: Game-Theoretic Alignment of LLM Assistants for Mutual Welfare

Siqi Zhu, David Zhang, Pedro Cisneros-Velarde, Jiaxuan You

Keywords: Human-Centered AI, Game Theory, Large Language Model, Reinforcement Learning

TL;DR: We train the LLM assistant through reinforcement learning guided by game-theoretic principles.

Abstract: Large Language Models (LLMs) have achieved remarkable progress in reasoning, yet often behave irrationally in tasks such as writing, information seeking, or providing practical guidance. Conventional alignment practices typically assume that maximizing model reward also maximizes user welfare, but this assumption frequently fails in practice: models may over-clarify or generate overly verbose reasoning when users prefer concise answers. Such behaviors resemble the prisoner’s dilemma, where individually rational choices lead to socially suboptimal outcomes. The fundamental challenge is that LLMs lack a principled mechanism for mutually beneficial decision making. We propose Game-Theoretic Alignment ($\textbf{GTAlign}$), an alignment framework that integrates game-theoretic decision making into both reasoning and training. During reasoning, the model explicitly treats user LLM interaction as a strategic game: it constructs payoff matrices within its reasoning chain to estimate welfare for both itself and the user, and then selects actions that are mutually beneficial. During training, we introduce a mutual welfare reward that reinforces cooperative responses, aligning model behavior with socially efficient outcomes. In addition, we introduce an inference technique that leverages game-theoretic reasoning to dynamically adapt LLM's response when payment modes of LLM service change. Extensive experiments demonstrate that GTAlign substantially improves reasoning efficiency, answer quality, and mutual welfare over baseline methods across diverse tasks.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 1945