UnHIDE: A Novel Framework for Unsupervised Human-Interpretable Dialogue Exploration

Patrícia Ferreira, Ana Alves, Catarina Silva, Hugo Gonçalo Oliveira

Abstract: Dialogue systems are increasingly central to applications in customer service, virtual assistance, and beyond, generating vast amounts of conversational data. While these systems have advanced with the exploitation of large language models (LLMs), they still face key limitations, some, in fact, strengthened by the black-box nature of such models, including the lack of feedback mechanisms and the absence of effective solutions for human-in-the-loop interaction and iterative improvement. As a result, understanding, refining, and debugging dialogue behavior remains a major challenge. To address this, we introduce UnHIDE, a novel, unsupervised framework for Human-Interpretable Dialogue Exploration. UnHIDE is designed to support human understanding of large collections of dialogues by surfacing interpretable structures and trends. It operates in three stages: 1) utterance clustering to group semantically similar dialogue turns, 2) flow discovery to build dialogue trajectories based on these clusters, and 3) the computation of interpretable metrics to analyze flow complexity, sentiment progression, and response times. We evaluate UnHIDE using a newly-created, automatically-generated, task-oriented dialogue dataset, where dialogue length, sentiment dynamics, and timing are systematically varied. Our results show that UnHIDE reliably captures these variations and provides actionable insights into dialogue structure and quality. By enabling transparent, human-interpretable analysis of dialogue without supervision, UnHIDE offers a powerful tool for diagnosing and improving dialogue systems. It not only fills a critical gap in feedback and interpretability, but also lays the groundwork for incorporating human-in-the-loop practices into future conversational Artificial Intelligence (AI) development.