Integrating Vectorized Lexical Constraints for Neural Machine Translation

Anonymous

Abstract: Lexically constrained neural machine translation (NMT), which controls the generation of NMT models with pre-specified constraints, is important in many practical scenarios. Due to the representation gap between discrete constraints and continuous vectors of NMT models, most existing works propose to construct synthetic data or modify the decoding algorithm to impose lexical constraints, treating the NMT model as a black box. In this work, we directly integrate the constraints into NMT models through vectorizing discrete constraints into continuous keys and values that can be utilized by the attention modules of NMT models. The proposed integration method is based on the assumption that the correspondence between the keys and values in attention modules is naturally suitable for modeling constraint pairs. Experimental results show that our method consistently outperforms several representative baselines on four language pairs, demonstrating the necessity of integrating vectorized lexical constraints.