Meta Back-Translation

Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig

Keywords: meta learning, machine translation, back translation

Abstract:

Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data. However, several recent works have found that better translation quality in the pseudo-parallel data does not necessarily lead to a better final translation model, while lower-quality but diverse data often yields stronger results instead.
In this paper we propose a new way to generate pseudo-parallel data for back-translation that directly optimizes the final model performance. Specifically, we propose a meta-learning framework where the back-translation model learns to match the forward-translation model's gradients on the development data with those on the pseudo-parallel data. In our evaluations in both the standard datasets WMT En-De'14 and WMT En-Fr'14, as well as a multilingual translation setting, our method leads to significant improvements over strong baselines.

One-sentence Summary: Use meta learning to teach the back-translation model to generate better back-translated sentences.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Code: [ google-research/google-research](https://github.com/google-research/google-research)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2102.07847/code)