Variational Attention for Commonsense Knowledge Aware Conversation Generation

Guirong Bai, Shizhu He, Kang Liu, Jun Zhao

Abstract: Conversation generation is an important task in natural language processing, and commonsense knowledge is vital to provide a shared background for better replying. In this paper, we present a novel commonsense knowledge aware conversation generation model, which adopts variational attention for incorporating commonsense knowledge to generate more appropriate conversation. Given a post, the model retrieves relevant knowledge graphs from a knowledge base, and then attentively incorporates knowledge to its response. For enhancing attention to incorporate more clean and suitable knowledge into response generation, we adopt variational attention rather than standard neural attention on knowledge graphs, which is unlike previous knowledge aware generation models. Experimental results show that the variational attention based model can incorporate more clean and suitable knowledge into response generation.