Mindmaster Roleplay: A Social Reasoning and Planning Benchmark

ICLR 2026 Conference Submission16192 Authors

Keywords: social intelligence, social reasoning, social planning, multi-agent, mental state, mind inference, theory of mind, cognitive architecture, belief, intent, value, social interaction, decision-making

TL;DR: We provide an platform for dyadic social interaction, and collect a valued dataset with fine-grained first-person mental state annotations. Our experiments and analyses with LLMs and human reveal intriguing phenomena.

Abstract: Social intelligence is one of the most challenging capabilities to develop in AI systems. Existing benchmarks for social reasoning mainly rely on unstructured text dialogues or simplified scenarios. There are very limited platforms that can support the community to systematically investigate the complex social cognitive mechanisms in social interactions. Thus, we present Mindmaster Roleplay, a social interaction platform that captures the dynamic interplay between beliefs, intentions, values, and actions through dyadic role-play games. Our platform provides interpretable first-person annotations of mental states, enabling researchers to trace how reasoning evolves and influences decision-making in diverse social scenarios. Our dataset establishes a valuable foundation for training and evaluating AI systems that more closely resemble human social intelligence in complex social reasoning tasks. Our experiments and analyses with both LLMs and human participants reveal a range of intriguing phenomena in social reasoning and decision-making. We will release our platform, dataset, code, and models upon acceptance.

Primary Area: datasets and benchmarks

Submission Number: 16192