Moment matching training for neural machine translation: An empirical study

Long H. B. Nguyen, Nghi T. Pham, Le D. C. Duc, Cong Duy Vu Hoang, Dien Dinh

Abstract: In recent years, Neural Machine Translation (NMT), which harnesses the power of neural networks, has achieved astonishing achievements. Despite its promise, NMT models can still not model prior external knowledge. Recent investigations have necessitated the adaptation of past expertise to both training and inference methods, resulting in translation inference issues. This paper proposes an extension of the moment matching framework that incorporates advanced prior knowledge without interfering with the inference process by using a matching mechanism between the model and empirical distributions. Our tests show that the suggested expansion outperforms the baseline and effectively over various language combinations.