Rewriter-Evaluator Framework for Neural Machine Translation

Yangming Li, Kaisheng Yao

Keywords: Neural Machine Translation, Post-editing mechanism, Polish Mechanism, Proper Termination Policy

Abstract: Encoder-decoder architecture has been widely used in neural machine translation (NMT). A few methods have been proposed to improve it with multiple passes of decoding. However, their full potential is limited by a lack of appropriate termination policy. To address this issue, we present a novel framework, Rewriter-Evaluator. It consists of a rewriter and an evaluator. Translating a source sentence involves multiple passes. At every pass, the rewriter produces a new translation to improve the past translation and the evaluator estimates the translation quality to decide whether to terminate the rewriting process. We also propose a prioritized gradient descent (PGD) method that facilitates training the rewriter and the evaluator jointly. Though incurring multiple passes of decoding, Rewriter-Evaluator with the proposed PGD method can be trained with similar time to that of training encoder-decoder models. We apply the proposed framework to improve the general NMT models (e.g., Transformer). We conduct extensive experiments on two translation tasks, Chinese-English and English-German, and show that the proposed framework notably improves the performances of NMT models and significantly outperforms previous baselines.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: We propose a novel framework, Rewriter-Evaluator, that achieves proper termination policy for multi-pass decoding.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2012.05414/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=fyuIxQIvje