RoleInstructor: A user-centered plug-and-play method for enhancing the interaction performance of LLM-based intelligent NPC systems

ACL ARR 2025 May Submission5513 Authors

Abstract: We present RoleInstructor, a novel user-centered plug-and-play method designed to address the critical challenge of maintaining long-term conversational coherence and user engagement in LLM-driven Intelligent non-player character (NPC) systems. Unlike existing approaches that rely on pre-defined heuristics or indirect optimization objectives, our method directly extracts human-evaluative metrics from player feedback data through a fine-grained annotation pipeline. These metrics are then operationalized via a two-stage optimization process: (1) metric-aware model calibration using constrained fine-tuning, and (2) dynamic adaptation through contextual instruction during inference. Through experimental validation across three distinct LLM-based NPC architectures, we demonstrate that RoleInstructor achieves statistically significant improvements by an average of 36.9% when deployed across heterogeneous NPC systems. Our code, dataset, and models will be released on GitHub.

Paper Type: Long

Research Area: Human-Centered NLP

Research Area Keywords: human-AI interaction/cooperation, human-centered evaluation, user-centered design;

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data resources

Languages Studied: Chinese，English

Submission Number: 5513