Terminology-Enriched Meta-curriculum Learning for Domain Neural Machine Translation

Zheng Chen, Yifan Wang

Abstract: As a data-driven approach, neural machine translation (NMT) heavily depends on large parallel corpora. Nevertheless, such corpora are frequently unavailable in domains of user interest, consequently diminishing the performance of NMT models in terms of domain robustness and adaptability. To tackle this challenge, this paper presents an innovative training method for multi-domain translation that leverages meta-curriculum learning and terminology information. To utilize domain-specific terminology, the proposed method first need to extract, align, and filter terms, thereby integrating bilingual terminology into the training dataset. Following this, the aligned sentences are sorted according to the domain similarity scores with the general domain in a curriculum learning manner. Then, the training data is divided into sub-datasets in ascending order of difficulty. A meta-learning technique is then employed to train the model utilizing these partitioned training datasets as tasks, ultimately yielding a translation model with exceptional domain robustness and remarkable domain adaptability. Experimental results on test data from both seen and unseen domains demonstrate that the proposed method yields an average improvement of 2.44 in BLEU scores on the test sets of multiple domains compared to the pre-training and fine-tuning method, and a 1.54 BLEU scores increase over the meta-curriculum learning approach without terminology information. Upon fine-tuning with a small amount of target domain data, the proposed method outperforms these two baselines by 2.62 and 1.5 in BLEU scores, respectively. These outcomes underscore the efficacy of the proposed method in improving NMT performance in scenarios with limited domain-specific data.