Latent Topic Conversational Models

Tsung-Hsien Wen, Minh-Thang Luong

Abstract: Despite much success in many large-scale language tasks, sequence-to-sequence (seq2seq) models have not been an ideal choice for conversational modeling as they tend to generate generic and repetitive responses. In this paper, we propose a Latent Topic Conversational Model (LTCM) that augments the seq2seq model with a neural topic component to better model human-human conversations. The neural topic component encodes information from the source sentence to build a global “topic” distribution over words, which is then consulted by the seq2seq model to improve generation at each time step. The experimental results show that the proposed LTCM can generate more diverse and interesting responses by sampling from its learnt latent representations. In a subjective human evaluation, the judges also confirm that LTCM is the preferred option comparing to competitive baseline models.

TL;DR: Latent Topic Conversational Model, a hybrid of seq2seq and neural topic model to generate more diverse and interesting responses.

Keywords: conversational modeling, dialogue, chitchat, open-domain dialogue, topic model, neural variational inference, human evaluation, latent variable model, gaussian reparameterisation trick

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1809.07070/code)