LLM-Based Social Simulations Require a Boundary

Zengqing Wu, Run Peng, Takayuki Ito, Chuan Xiao

Keywords: LLM-based social simulation, boundaries, agent heterogeneity, behavioral alignment

TL;DR: We argue that LLM-based social simulations should establish applicable boundaries to enhance its contribution to social science research.

Abstract: This work argues that large language model (LLM)-based social simulations must establish clear boundaries to meaningfully contribute to social science research. While LLMs offer promising capabilities for modeling human-like agents compared to traditional agent-based modeling, they face fundamental limitations that constrain their reliability for social pattern discovery. The core issue lies in LLMs' tendency toward an "average persona" that lacks sufficient behavioral heterogeneity, a critical requirement for simulating complex social dynamics. We examine three key boundary problems: alignment (simulated behaviors matching real-world patterns), consistency (maintaining coherent agent behavior over time), and robustness (reproducibility under varying conditions). We propose heuristic boundaries for determining when LLM-based simulations can reliably advance social science understanding. Our analysis reveals that these simulations are most valuable when focusing on collective patterns rather than individual trajectories, when agent behaviors align with real population averages despite limited variance, and when proper validation methods confirm simulation robustness. We provide a practical checklist to guide researchers in determining the appropriate scope and claims for LLM-based social simulations.

Primary Area: generative models

Submission Number: 11711