Dialogue Response Evaluation Model with Conversational Feature Sensitive Negative Sampling

Dongjun Kang, JinYeong Bak

Abstract: Evaluating the conversational responses is a challenging task. This is because there are so many possible responses in open-domain conversation. Recent work finds that appropriate negative samples are effective in practice, but there was problem that labeled conversational negative sample was insufficient. It is important to create an appropriate negative sample automatically because it is too costly to create all possible responses by person and annotate the response’s coherence score. To address this problem, we propose a method for generating and labeling feature sensitive negative responses for conversation automatically. Besides, we show that the model learned with the generated negative sample performs well with high correlation between human score and model score.