Inducing Grammars with and for Neural Machine Translation

Ke Tran, Yonatan Bisk

Abstract: Previous work has demonstrated the benefits of incorporating additional linguistic annotations such as syntactic trees into neural machine translation. However the cost of obtaining those syntactic annotations is expensive for many languages and the quality of unsupervised learning linguistic structures is too poor to be helpful. In this work, we aim to improve neural machine translation via source side dependency syntax but without explicit annotation. We propose a set of models that learn to induce dependency trees on the source side and learn to use that information on the target side. Importantly, we also show that our dependency trees capture important syntactic features of language and improve translation quality on two language pairs En-De and En-Ru.

TL;DR: improve NMT with latent trees

Keywords: structured attention, neural machine translation, grammar induction

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1805.10850/code)