Dialogue Representation Learning: A New Benchmark and Weighted Contrastive Learning Approach

Anonymous

Abstract: High-quality pre-trained text representations are powerful tools for various downstream tasks. However, dialogue representation learning has received less attention compared to tasks such as sentence representation learning. This can be attributed to two main challenges: 1) the lack of standard evaluation benchmarks on dialogue representation learning, and 2) the complexity of incorporating dialogue corpus into existing representation learning paradigms. To overcome these challenges, we present the first comprehensive evaluation benchmark called \textbf{DiaEval} (\textbf{Dia}logue Representation \textbf{Eval}uation Benchmark), which covers 5 datasets across 3 tasks including action prediction, dialogue inference, and response retrieval. These datasets are meticulously selected to ensure their comprehensiveness and representativeness. Second, we propose a new dialogue embedding method called \textbf{WMDC} (\textbf{W}eighted \textbf{M}ulti-window-sized \textbf{D}ialogue \textbf{C}ontrastive learning). WMDC leverages multiple context windows and sample reweighting with contrastive learning to obtain universal dialogue embeddings. The use of multiple context windows allows flexible encoding with multiple granularity while the reweighting method addresses the anisotropy and lack of informativeness issues within the learned dialogue embedding space. Through extensive comparison with various competitive baselines, WMDC achieves state-of-the-art performance on all tasks demonstrating its effectiveness and scalability.

Paper Type: long

Research Area: Semantics: Sentence-level Semantics, Textual Inference and Other areas

Contribution Types: Publicly available software and/or pre-trained models, Data resources

Languages Studied: English