Good Meta-tasks Make A Better Cross-lingual Meta-transfer Learning for Low-resource Languages

Linjuan Wu, Zongyi Guo, Baoliang Cui, Haihong Tang, Weiming Lu

Submission Type: Regular Long Paper

Submission Track: Multilinguality and Linguistic Diversity

Keywords: Few-shot Cross-lingual Transfer Learning, Low-resource Languages, Model-agnostic Meta-learning

Abstract: Model-agnostic meta-learning has garnered attention as a promising technique for enhancing few-shot cross-lingual transfer learning in low-resource scenarios. However, little attention was paid to the impact of data selection strategies on this cross-lingual meta-transfer method, particularly the sampling of cross-lingual meta-training data (i.e. meta-tasks) at the syntactic level to reduce language gaps. In this paper, we propose a Meta-Task Collector-based Cross-lingual Meta-Transfer framework (MeTaCo-XMT) to adapt different data selection strategies to construct meta-tasks for meta-transfer learning. Syntactic differences have an effect on transfer performance, so we consider a syntactic similarity sampling strategy and propose a syntactic distance metric model consisting of a syntactic encoder block based on the pre-trained model and a distance metric block using Word Move's Distance (WMD). Additionally, we conduct experiments with three different data selection strategies to instantiate our framework and analyze their performance impact. Experimental results on two multilingual NLP datasets, Wikiann and TydiQA, demonstrate the significant superiority of our approach compared to existing strong baselines.

Submission Number: 1771