Do biliguanality and orality matter ? Comparative fine-tuning roBERTa and XLM-RoBERTa for dialog act classification

Loris Bulliard, Arthur Katossky

Abstract: The task of classifying dialog act intents is crucial in the development of intelligent dialog systems as it helps to guide the generation of responses and avoid generic, unnatural replies. In this study, we study how the inclusion of (a) oral and (b) multilingual examples in a pre-training phase may improve downstream intent classification. We benchmarks model variants based on roBERTa and XLM- roBERTa and pre-trained on OpenSubtitles by masked language modeling. We then perform dialog act classification on Maptask (English) and Loria (French) datasets from the MIAM collection, and demonstrate the accuracy improvements achieved through this method.