Lost in Literalism: How Supervised Training Shapes Translationese in LLMs

ACL ARR 2024 December Submission387 Authors

Abstract:

Large language models (LLMs) have achieved remarkable success in machine translation, demonstrating impressive performance across diverse languages. However, translationese—characterized by overly literal and unnatural translations—remains a persistent challenge in LLM-based translation systems. Despite their pre-training on vast corpora of \textit{natural} utterances, LLMs exhibit translationese errors and generate unexpected \textit{unnatural} translations, stemming from biases introduced during supervised fine-tuning (SFT). In this work, we systematically evaluate the prevalence of translationese in LLM-generated translations and investigate its roots during supervised training. We introduce methods to mitigate these biases, including polishing golden references and filtering unnatural training instances.
Empirical evaluations demonstrate that these approaches significantly reduce translationese while improving translation naturalness, validated by human evaluations and automatic metrics. Our findings highlight the need for training-aware adjustments to optimize LLM translation outputs, paving the way for more fluent and target-language-consistent translations.

Paper Type: Long

Research Area: Machine Translation

Research Area Keywords: machine translation

Languages Studied: English, German, Chinese, Russian, Czech, Icelandic

Submission Number: 387