That is a good looking car !: Visual Aspect based Sentiment Controlled Personalized Response Generation

Anonymous

Abstract: In a conversational system, generating utterances that communicate consistent and relevant preferences is vital for more personalized conversations. In this paper, we propose a task of generating utterances grounded on some assigned aspect-preferences-profile. These aspect-preference profiles consist of a list of aspect-sentiment tuples, denoting the preference of the speaker for some aspect in the form of sentiment ("positive" or "negative"). Since no prior dataset containing such profiles is available, we enhance Image-Chat data by assigning these profiles to each user in a conversation. The conversations in this dataset are based on an image, therefore the aspects are present in images as well as dialogue history. We build a BERT and ResNet-based encoder-decoder model with a memory network to store preference-profile. Through our experiments, we show that our model can generate responses that convey the sentiment of relevant aspects in accordance with the assigned profile. Both automatic and manual evaluations show the effectiveness of our model and dataset. Our proposed system when using these profiles achieves a BLEU-1 score of 15.93 on this new task, which is an improvement of 2.92 points from the baseline experiment that does not use aspect-preference profiles.

Paper Type: long