Improving Personalized Dialogue Generation Models with Data-level Distillation and Diversification

Anonymous

Abstract: Personalized dialogue generation is a challenging task in which a persona-consistent response needs to be generated conditioning both persona texts and dialogue utterances, being more complex than conventional dialogues. Multiple persona texts and utterances exist in one sample and some of them can be distractors for generating. Thus even strong models have difficulty posing attention to suitable personas so generating persona-irrelevant responses. Besides, the limited data scale and diversity further affect the performance. Thus, we start from data and propose to boost the model in data-level distillation and diversification (D$^3$). We first distill the original training samples into simplified persona-consistent ones, lowering the difficulty by removing redundant information in personas and dialogue history. Next in the diversification, we increase both the amount and diversity of distilled data to ease its insufficiency. A model will be trained via curricula, first on easier augmented samples and then on harder original ones. Experiments on the PersonaChat benchmark dataset illustrate the superiority of our method when packed with two strong base dialogue models (Transformer and GPT2) on various automatic metrics and human evaluation.