True Bilingual NMT

Mohamed Anwar, Lekan Raheem, Maab Elrasheed, Melvin Johnson, Julia Kreutzer

Keywords: Machine Translation, Neural Machine Translation, NMT, Code-Mixing, Code-Switching, Code-Switch Translation, Multilingual Machine Translation, Transformer, WMT14, newstest2008, newstest2014, alignment loss, Joint Learning, English-French Translation

TL;DR: We are proposing an NMT model that is able to translate in both directions of the language-pair it was trained on; and also translate code-switched sentence that outperforms all baselines.

Abstract: Bilingual machine translation permits training a single model that translates monolingual sentences from one language to another. However, a model is not truly bilingual unless it can translate back and forth in both language directions it was trained on, along with translating code-switched sentences to either language. We propose a true bilingual model trained on WMT14 English-French (En-Fr) dataset. For better use of parallel data, we generated synthetic code-switched (CSW) data along with an alignment loss on the encoder to align representations across languages. Our model strongly outperforms bilingual baselines on CSW translation while maintaining quality for non-code switched data.