Trust but Check: LLM-Assisted Review of Human Translations in African Languages

Tadesse Destaw Belay, Henok Biadglign Ademtew, Idris Abdulmumin, Sukairaj Hafiz Imam, Abubakar Juma Chilala, Godfred Agyapong, CHINEDU EMMANUEL MBONU, Basil Friday Ovu, Catherine Nana Nyaah Essuman, Alfred Malengo Kondoro, Sonia Adhiambo, Daud Abolade, Ponts'o Mpholle, Nicholaus Dismas Ladislaus, Saminu Mohammad Aliyu, Gali Ahmad Samuel, Fabrice Hakuzimana, Mike Nzirainengwe, Temitayo Olatoye, Sileshi Bogale Haile et al. (7 additional authors not shown)

Abstract:

Large-scale translation projects for low-resource languages mostly rely on human translators to ensure cultural and linguistic fidelity.
However, even professionally produced translations often contain subtle translation errors that are difficult to detect. Manual quality control at scale becomes prohibitively expensive, creating a major bottleneck in the development of high-quality Natural Language Processing (NLP) resources. Recent advances in multilingual large language models (LLMs) offer promising support for annotation workflows with human-in-the-loop settings. In this work, we investigate the use of LLMs to assist in auditing translation quality, enabling more efficient quality control pipelines for low-resource African languages. We audit translations in 11 African languages using the MAFAND-MT dataset, combining LLM-as-a-judge, native-speaker human review, and automated metrics. Our quality-audited version of MAFAND-MT test set yields performance gains across all languages, with BLEU scores ranging from 0.4 to 9.27 and chrF scores ranging from 0.3 to 8.69. Our findings further indicate that state-of-the-art LLMs, such as GPT-5.1, can assist in auditing translation quality and suggesting candidate corrections for low-resource languages. However, they remain far from being a stand-alone solution for the automatic correction of human translations in African languages.

Submission Number: 50