Go to ACL ARR 2024 August homepageInvestigating the Effect of Mental Models in User Interaction with an Adaptive Dialog Agent
Abstract: Mental models play an important role in whether user interaction with intelligent systems, such as dialog systems is successful or not.
Adaptive dialog systems present the opportunity to align a dialog agent's behavior with heterogeneous user expectations.
However, there has been little research into what mental models users form when interacting with a task-oriented dialog system, how these models affect users' interactions, or what role system adaptation can play in this process, making it challenging to avoid damage to human-AI partnership.
In this work, we collect a new publicly available dataset for exploring user mental models about information seeking dialog systems.
We demonstrate that users have a variety of conflicting mental models about such systems, the validity of which directly impacts the success of their interactions and perceived usability of system.
Furthermore, we show that adapting a dialog agent's behavior to better align with users' mental models, even when done implicitly, can improve perceived usability, dialog efficiency, and success.
To this end, we argue that implicit adaptation can be a valid strategy for task-oriented dialog systems, so long as developers first have a solid understanding of users' mental models.
Paper Type: Long
Research Area: Dialogue and Interactive Systems
Research Area Keywords: Dialogue and Interactive Systems, Computational Social Science and Cultural Analytics, Resources and Evaluation
Contribution Types: NLP engineering experiment, Data resources, Data analysis
Languages Studied: English
Submission Number: 191
Loading