"Mm, Wat?" Detecting Other-intiated Repair Requests in Dialogue

ACL ARR 2025 May Submission7343 Authors

Abstract: Maintaining mutual understanding is a key component in human-human conversation to avoid conversation breakdowns, in which repair, particularly Other-Initiated Repair (OIR, when one speaker signals trouble and prompts the other to resolve), plays a vital role. However, Conversational Agents (CAs) still fail to recognize user-initiated repair requests, leading to breakdowns or disengagement. This work proposes a multimodal approach to automatically detect OIR requests in Dutch dialogues by integrating linguistic and prosodic features grounded in Conversation Analysis. The results show that prosodic cues complement linguistic features and significantly improve the results of pre-trained text and audio embeddings, offering insights into how different features interact. Future directions include incorporating visual cues, exploring large language models (LLMs), and applying the model in CA systems.

Paper Type: Long

Research Area: Discourse and Pragmatics

Research Area Keywords: discourse relations, dialogue, conversation

Contribution Types: Model analysis & interpretability

Languages Studied: Dutch

Submission Number: 7343