Eliciting Human-like Social Reasoning in Large Language Models

ICLR 2026 Conference Submission16651 Authors

Keywords: Large Language Model, Social Simulation, LLM Reasoning

Abstract:

Large language models (LLMs) have gained significant attention for their potential to replicate human participants in social science simulations.
However, previous works on LLM reasoning focus on enhancing the capabilities for math and logical problems, overlooking the reasoning process behind social behavior, such as controversial social attitudes, moral dilemmas, and economic games.
In this study, we explore the limitations of current models and propose a new approach to improve their human-likeness in social behavioral reasoning tasks.
We introduce the Social-Behavioral-Reasoning (SBR) dataset, comprising 1,560 quadruples of human profiles, social questions, reasoning processes, and final choices.
Utilizing this dataset, we evaluate large reasoning models (LRMs), revealing a contradiction: while LRMs increase society-level diversity, they fail to maintain individual-level accuracy.
Our findings further indicate that the observed increase in diversity is primarily attributed to random variation introduced by longer reasoning durations, rather than improved understanding of human diversity.
To address these issues, we propose the Reasoning-Enhanced-SFT method, which explicitly aligns both the reasoning and final choices with human data.
Our experimental results demonstrate that our method significantly improves both in-domain and out-of-domain performance, enhancing the generalization ability across diverse social contexts.
Our user study results confirm the model's ability to produce a reasoning process more closely aligned with specific human reasoning patterns.
Our work offers a new pathway to overcome the challenges that limit the use of LLMs in social simulations. Aligning model outputs with human reasoning boosts LLMs' credibility and applicability in social science, enabling more precise and insightful simulations of human behavior.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 16651