Document-Level Neural Machine Translation With Document Embeddings

Linxing Zhu, Shu Jiang, Hai Zhao, Zuchao Li, Jiashuang Huang, Weiping Ding, Bao-Liang Lu

Abstract: Standard neural machine translation (NMT) assumes that document-level context information is irrespective. Most existing document-level NMT methods are satisfied with a smattering sense of shallow document-level information, such as using a few context sentences surrounding the source sentence as document-level information. Our work focuses on exploiting detailed document-level context in terms of multiple forms of document embeddings, which can sufficiently model deeper and richer document-level context. The proposed document-level NMT is implemented to enhance the Transformer baseline by introducing both global and local document-level clues on the source end. We compress the entire document text with explicit boundaries into a token-size global static document embedding, and the neighboring sentences as a token-size local dynamic document embedding and concatenate with the source tokens. Experiments reveal that the proposed method significantly improves the translation performance over strong baselines and other related studies.

External IDs:doi:10.1109/access.2025.3568089