Augmenting Memory Networks for Rich and Efficient Retrieval in Grounded Dialogue

Anonymous

Abstract: Grounded dialogue consists of conditioning a conversation on additional latent inputs ("factoids") beyond the dialogue context, such as Wikipedia articles, IMDB reviews, persona, and images. Due to a scarcity of <context, factoid> labels, it is common practice to jointly learn the knowledge-selection and grounded response generation tasks end-to-end. When conditioning the response on these factoids, previous work has either treated the factoids as a weighed average vector, or separately computed probabilities for each <context, factoid> pair. However, the former creates a bottleneck whilst the latter prevents factoids from being considered jointly. Our new method, PolyMemNet, learns a matrix representation of the context and factoids, allowing for multiple factoids to be jointly considered in response selection, without imposing a bottleneck. We show how this achieves up to a $17\%$ boost in knowledge-selection accuracy and $13\%$ in response-selection accuracy versus memory networks.