Beyond Parallel Corpora: Assessing LLMs and State-of-the-Art Models for Specialised Texts Translation and Error Detection

ACL ARR 2025 February Submission3195 Authors

Abstract: MT of specialised texts poses particular challenges due to domain-specific terminology, phraseology and structural conventions. LLMs offer a promising alternative to traditional MT approaches, especially in domains with limited parallel corpora (specialised texts being a notable example). However, their performance remains underexplored, despite the fact that this type of translation has significant socio-economic implications. In this study, we evaluate the ability of LLMs and state-of-the-art translation models to translate specialised texts, using an error typology designed for the evaluation of specialised translation to provide a qualitative assessment. Our approach provides detailed insights into translation challenges and investigates whether LLMs can also detect errors in LSP translations.

Paper Type: Short

Research Area: Machine Translation

Research Area Keywords: language with specific purpose, machine translation evaluation, error detection in MT, prompts

Contribution Types: NLP engineering experiment

Languages Studied: French, English

Submission Number: 3195