An Analysis of Stopping Strategies in Conversational Search Systems
Keywords: Dialogue Systems, User Modeling, Stopping, Conversational Search
Abstract: Stopping strategies are a crucial aspect of conversational systems and user simulations, as they provide insight into when users end their interactions, which is vital for creating realistic simulations. While the Information Retrieval (IR) community has studied this topic extensively, little research has been done on stopping strategies in Conversational Search Systems (CSSs).
This is due to conversations' unique sequential and interactive nature, where traditional IR techniques struggle to accurately predict stopping points well and require new methods to be adapted from traditional IR techniques.
In this paper, we adapt Stopping Rules (SRs) from the IR community to the conversational setting, creating new SRs and identifying core features for each.
We then analyze these features with several conversational datasets and aim to identify key features that predict stopping points in conversations between users and CSSs.
We found that models based on these features performed well in predicting stopping points and that textual statistical features, i.e., numbers of words, nouns, noun phrases and sentences users received from systems or outputted by users, always play a significant role in determining stopping points, with the number of outputted \textit{unique nouns} playing a particularly important role as an SR.
Our results provide a foundation for developing more realistic user models and simulators and guiding the design of more reliable evaluation measures for CSSs.
Submission Number: 35
Loading