TaskSim: A Similarity Metric for Task-Oriented Conversations

Anonymous

Abstract: The popularity of conversational digital assistants has resulted in efforts to improve user experience by extracting insights from the logs. These approaches utilize distance based metrics to identify similarities between user conversations. These metrics are typically designed to compare text snippets and do not take advantage of the unique conversational features in dialogues that are absent from other textual sources. To address this gap, in this work, we present TaskSim, a novel conversational similarity metric that utilizes different dialogue components (e.g.utterances, intents, and slots) with optimal transport. Extensive experimental evaluation of the TaskSim metric on a benchmark dataset demonstrate its better performance over other traditional similarity approaches.

Paper Type: short

Research Area: Dialogue and Interactive Systems