Beyond English: Examining the Impact of Prompt Translation Strategies in Multilingual Natural Language Tasks
Abstract: Despite advances in the multilingual capabilities of Large Language Models (LLMs) across diverse Natural Language Processing (NLP) tasks, English remains the dominant language for LLM research and development. This
has led to the widespread practice of pre-translation, i.e., translating the task prompt into English before inference. Selective pre-translation, a more surgical approach, focuses on translating specific prompt components. However, its current use lacks a systematic research foundation. Consequently, the optimal pre-translation strategy for various multilingual settings and tasks remains unclear. In this work, we aim to uncover the optimal setup for pre-translation by systematically assessing its modes of use. Specifically, we view the prompt as a modular entity, composed of four functional parts: instruction, context, examples (zero-shot / few-shot), and output, either of which could be translated or not. We evaluate pre-translation strategies across 35 languages covering both low and high-resource languages, and assessing various capabilities including Question Answering (QA), Natural Language Inference (NLI), Named Entity Recognition (NER), and Abstractive Summarization. Our experiments uncover the impact of factors as translation quality, similarity to English, and the size of pre-trained data, on the model performance with pre-translation. Finally, we suggest practical guidelines for choosing the optimal strategy in various multilingual scenarios.
Paper Type: Long
Research Area: Multilingualism and Cross-Lingual NLP
Research Area Keywords: Efficient/Low-Resource Methods for NLP, Language Modeling, Multilingualism and Cross-Lingual NLP
Contribution Types: Model analysis & interpretability, Approaches to low-resource settings
Languages Studied: Arabic, Bulgarian, Chinese, German, Greek, Hindi, Spanish, Swahili, Thai, Turkish, Urdu, Arabic, German, Greek, Romanian, Russian, Vietnamese, Assamese, Bengali, Hindi, Malayalam, Telugu, Bambara, Ese, Hausa, Yoruba, Chinese, French, Italian, Portuguese, Serbian, Slovak, Swedish, Azerbaijani, French, Japanese, Korean, Nepali, Persian, Portuguese, Spanish, Turkish, Uzbek
Submission Number: 5124
Loading