Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance

Anonymous

Paper Link: https://openreview.net/forum?id=myR1JFrsAyf

Paper Type: Long paper (up to eight pages of content + unlimited references and appendices)

Abstract: Human-translated text displays distinct features from naturally written text in the same language. This phenomena, known as translationese, has been argued to confound the machine translation (MT) evaluation. Yet, we find that existing work on translationese neglects some important factors and the conclusions are mostly correlational but not causal. In this work, we collect CausalMT, a dataset where the MT training data are also labeled with the human translation directions. We inspect two critical factors, the train-test direction match (whether the human translation directions in the training and test sets are aligned), and data-model direction match (whether the model learns in the same direction as the human translation direction in the dataset). We show that these two factors have a large causal effect on the MT performance, in addition to the test-model direction mismatch highlighted by existing work on the impact of translationese. In light of our findings, we provide a set of suggestions for MT training and evaluation. Our code and data are at https://github.com/EdisonNi-hku/CausalMT

Presentation Mode: This paper will be presented in person in Seattle

Virtual Presentation Timezone: UTC+1

Copyright Consent Signature (type Name Or NA If Not Transferrable): Zhijing Jin

Copyright Consent Name And Address: Max Planck Institute for Intelligent Systems, Tuebingen, Germany (Max-Planck-Ring 4, Tuebingen, 72076, Germany)