Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation
Submission Type: Regular Long Paper
Submission Track: Machine Translation
Submission Track 2: Multilinguality and Linguistic Diversity
Keywords: Multilingual Machine Translation, Shared Vocabulary
TL;DR: We inject priors of meaning-equivalent words into embedding table to enhance knowledge transfer for multilingual machine translation.
Abstract: Using a shared vocabulary is common practice in Multilingual Neural Machine Translation (MNMT). In addition to its simple design, shared tokens play an important role in positive knowledge transfer, which manifests naturally when the shared tokens refer to similar meanings across languages. However, when words overlap is small, e.g., using different writing systems, transfer is inhibited. In this paper, we propose a re-parameterized method for building embeddings to alleviate this problem. More specifically, we define word-level information transfer pathways via word equivalence classes and rely on graph networks to fuse word embeddings across languages. Our experiments demonstrate the advantages of our approach: 1) the semantics of embeddings are better aligned across languages, 2) our method achieves evident BLEU improvements on high- and low-resource MNMT, and 3) only less than 1.0\% additional trainable parameters are required with a limited increase in computational costs, while the inference time is identical to baselines.
Submission Number: 2039
Loading