GRTr: Generative-Retrieval Transformers for Data-Efficient Dialogue Domain Adaptation

Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz

Abstract: Domain adaptation has recently become a key problem in dialogue systems research. Deep learning, while being the preferred technique for modeling such systems, works best given massive training data. However, in real-world scenarios, such resources are rarely available for new domains, and the ability to train with a few dialogue examples can be considered essential. Pre-training on large data sources and adapting to the target data has become the standard method for few-shot problems within the deep learning framework. In this paper, we present GRTr, a hybrid generative-retrieval model based on the large-scale general-purpose language model GPT[2] fine-tuned to the multi-domain metalwoz dataset. In addition to robust and diverse response generation provided by the GPT[2], our model is able to estimate generation confidence, and is equipped with retrieval logic as a fallback for the cases when the estimate is low. grtr is the winning entry at the fast domain adaptation task of DSTC-8 in human evaluation ($>$4% improvement over the 2nd place system). It also attains superior performance to a series of baselines on automated metrics on metalWOZ and multiWOZ, a multi-domain dataset of goal-oriented dialogues. In this paper, we also conduct a study of GRTr's performance in the setup of limited adaptation data, evaluating the model's overall response prediction performance on metalwoz and goal-oriented performance on multiWOZ.