CMM: Code-Switching with Manifold Mixup for Cross-Lingual Spoken Language Understanding

Tianjun Mao, Chenghong Zhang

Abstract: Spoken language understanding (SLU) is an important task which involves two subtasks, including intent detection and slot filling. Although it has achieved great success in high-resource languages, it still remains challenging in low-resource languages due to the lack of labeled training data. Consequently, there is growing interest in code-switching method for cross-lingual SLU to solve the problem in the low-resource languages. However, despite the success of existing models, most of these methods fail to effectively leverage the code-switched utterances. In this paper, we propose a novel framework termed CMM for zero-shot cross-lingual SLU which simplifies the learning task for the model. Specifically, we apply both mixup and curriculum learning method to dynamically combine the information from pure utterances and code-switched utterances. Experimental results demonstrate that the proposed framework improves the performance compared to several strong baselines and achieves the state-of-the-art performance on MultiATIS++ dataset, with a relative improvement of 3.0% in terms of overall accuracy over the previous best model.