Disentangling the Roles of Target-side Transfer and Regularization in Multilingual Machine Translation

Anonymous

Abstract: Multilingual Machine Translation (MMT) benefits from knowledge transfer across different language pairs. However, improvements in one-to-many translation are only marginal compared to many-to-one translation. A widely held assumption is that knowledge transfer barely plays a role in the target-side of MMT. The observed improvements in one-to-many MT are instead attributed to two possible reasons: increasing the amounts of source language data and target language regularization. In this paper, we conduct a large-scale study that varies the target-side languages along two dimensions, i.e., linguistic similarity and corpus size, to show the interplay between different factors (knowledge transfer, source data size, language regularization) for improving one-to-many translation. First, we find that positive knowledge transfer does occur on the target-side, which greatly benefits low- and medium-resource language pairs. Moreover, the performance discrepancy across different target languages also shows that increasing the source-side data cannot be the main reason for improving one-to-many MT. Furthermore, we show language regularization plays a crucial role in benefiting translation performance by enhancing the generalization ability and model inference calibration. We find a simple but effective way to utilize distant target data with the aim of regularizing the model, which surprisingly leads to translation performance gains.

Paper Type: long

Research Area: Machine Translation

Contribution Types: Model analysis & interpretability

Languages Studied: English;German;Spanish;Russian

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.