Differential-Aware Transformer for Partially Amended Sentence Translation

Takahiro Yamakoshi, Yasuhiro Ogawa, Katsuhiko Toyama

Abstract: We propose a Transformer-based differential translation architecture that targets statutory sentences partially modified by amendments. In translating post-amendment statutory sentences, translation focality—modifying only the amended expressions in the translation and retaining the others—is important to avoid misunderstanding the amendment’s contents. To sharpen the translation’s focality, we introduce a neural network architecture called a Copiable Translation Transformer that can copy expressions in the pre-amendment translated sentence as needed and generate expressions from the post-amendment original sentence. In experiments, we showed that our method outperformed the naive Transformer with a training corpus of partially amended sentences.