ChatNPC: Towards Immersive Video Game Experience through Naturalistic and Emotive Dialogue Agent

ACL ARR 2025 February Submission1965 Authors

Abstract: We present ChatNPC, a game companion designed to respond to players' subtle, unconscious comments and gestures during gameplay. The model integrates sequential data, benefiting from the dynamic nature of streams, including player information, spoken lines, and in-game context. It observes and apprehends players' emotional shifts and adjusts the responses accordingly. We leverage the recent progress in LLM's tool-calling capabilities to extract vital information from memory and recognize potential constraints for accurate reasoning to tackle the complexity of NPC conversation scenarios. The task is divided into: 1) a novel game sentinel agent (SenGent), 2) a memory capability, and 3) a chat planning tool for reasoning instantiation. The approach benefits from a lightweight game-template as an information framework, with relevant details and a thorough reasoning layout. We conduct extensive experiments on a newly developed game dataset for in-game context NPC dialogue and demonstrate that ChatNPC sequentially captures players' emotional shifts over time; responses are more naturalistic and human-like with appropriate conversational cues, pauses, and sighs; and utterances remain faithful to the dynamics of in-game context and player actions, supporting narrative continuity.

Paper Type: Long

Research Area: Dialogue and Interactive Systems

Research Area Keywords: spoken dialogue systems; human-in-the-loop; embodied agents; applications; conversational modeling;

Contribution Types: Reproduction study

Languages Studied: English

Submission Number: 1965