Template-augmented Dialogue Representation Learning

Minsik Oh, Guoyin Wang, Taiwoo Park, Puyang Xu

Abstract: Learning high-quality sentence embeddings from dialogue has drawn increasing attention as it is essential to solving various dialogue-oriented tasks with low annotation costs. However, directly gathering utterance relationships from conversations are difficult, while token-level annotations, \eg, entities, slots, and templates, are much easier to obtain. General sentence embedding methods are based on sentence-level self-supervised frameworks and cannot utilize token-level extra knowledge. In this paper, we introduce a new dialogue utterance embedding framework, \textbf{T}emplate-\textbf{a}ugmented \textbf{D}ialogue \textbf{S}entence \textbf{E}mbedding (TaDSE). This novel method utilizes template information to learn utterance representation effectively via a self-supervised contrastive learning framework. TaDSE augments each sentence with its corresponding template and then conducts pairwise contrastive learning over both sentence and template. We evaluate TaDSE performance on two downstream benchmark datasets. The experiment results show that TaDSE achieves significant improvements over previous SOTA methods. We further analyze the representation quality and show improved alignment and boosted local structure in semantic representation hyperspace.