Imitation Game: Toward Comprehensive Evaluation on Personalized Role-Playing on Social Media

ICLR 2026 Conference Submission18108 Authors

Keywords: Personalized Role-Playing, Social Media, Evaluation

Abstract: Social simulation observes the emergence of complex macro-scopic social patterns from individual interactions within a virtual social environment. A common practice in social simulation is to model individual as a statistical average of a specific group, failing to capture individual heterogeneity. To model such heterogeneity, we propose a personalized role-playing task in the context of social media, which provides environment for social simulation with massive authentic user interactions. As no public social media dataset concentrates especially on historical interactions of individual user for personalization, we collect data from reddit and construct our own dataset, consisting of 67 users, 7K posts, and 21K comments. And we introduce three key dimensions for personalized role-playing and conduct comprehensive evaluation on feasible role-playing methods. The results yield the following key findings:(1) existing methods struggle to achieve fine-grained personalized modeling; (2) merely scaling model parameters or applying reasoning models is insufficient to substantially enhance the level of personalization; (3) the evaluated methods exhibit significant vulnerability to noise within interaction context.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 18108