Agentic Persona Control and Task State Tracking for Realistic User Simulation in Interactive Scenarios

Hareeshwar Karthikeyan

Keywords: multi-agent environments, agentic orchestration, persona-driven simulation, task state tracking, interactive environments, human behavior simulation, dialogue environments, synthetic user data generation, behavioral attribute control, cognitive agent architectures, environment-based evaluation, agent adaption environment, ablation study, conversational agent testing, explainable agent behavior, explainable AI

TL;DR: A multi-agent framework decomposing human user simulation into specialized agents for state tracking and behavioral control achieves 2x better realism than single-LLM baselines in conversational AI testing.

Abstract: Testing conversational AI systems at scale across diverse domains necessitates realistic and diverse user interactions capturing a wide array of behavioral patterns. We present a novel multi-agent framework for realistic, explainable human user simulation in interactive scenarios, using persona control and task state tracking to mirror human cognitive processes during goal-oriented conversations. Our system employs three specialized AI agents: (1) a User Agent to orchestrate the overall interaction, (2) a State Tracking Agent to maintain structured task state, and (3) a Message Attributes Generation Agent that controls conversational attributes based on task progress and assigned persona. To validate our approach, we implement and evaluate the framework for guest ordering at a restaurant with scenarios rich in task complexity, behavioral diversity, and conversational ambiguity. Through systematic ablations, we evaluate the contributory efficacy of each agentic component to overall simulation quality in terms of persona adherence, task completion accuracy, explainability, and realism. Our experiments demonstrate that the complete multi-agent system achieves superior simulation quality compared to single-LLM baselines, with significant gains across all evaluation metrics. This framework establishes a powerful environment for orchestrating agents to simulate human users with cognitive plausibility, decomposing the simulation into specialized sub-agents that reflect distinct aspects of human thought processes applicable across interactive domains.

Archival Option: The authors of this submission want it to appear in the archival proceedings.

Submission Number: 141