Language Models as Proxies for Humans: Survey on LM- based User Simulation

UIUC Spring 2025 CS598 LLM Agent Workshop Submission11 Authors

Keywords: LLM, User Simulation, Human Metal Modeling, Survey

TL;DR: A survey on user simulation from classical approaches to use of language models

Abstract: User simulation has long been vital to AI research, enabling the development, training, and evaluation of interactive systems without constant human participation. The recent emergence of large language models (LLMs) has fundamentally transformed the capabilities and approaches to simulating user behavior. This paper presents the first comprehensive survey focused on LLM-based user simulation, examining how LLMs connect to and extend classical approaches. We propose a taxonomy that organizes the field along four key dimensions: Simulation Methodologies, User Behavior Modeling, Evaluation Frameworks, & Application Domains. Our analysis reveals how LLMs have enabled unprecedented advances in simulation fidelity, including more natural language variation, complex behavioral patterns, and cognitive modeling that was previously unattainable. We systematically compare LLM-based simulations with traditional rule-based and statistical approaches, highlighting their relative strengths and limitations across different application domains. This survey identifies unique challenges posed by LLM-based user simulation, including issues of controllability, alignment with specific user profiles, and the need for specialized evaluation metrics. Our work bridges classical and emerging approaches, providing researchers and practitioners with a unified framework for understanding and advancing this rapidly evolving field.

Submission Number: 11