Does the English Matter? Elicit Cross-lingual Abilities of Large Language Models

Leonardo Ranaldi, Giulia Pucci

Abstract:

Large Language Models reveal diverse abilities across different languages due to the disproportionate amount of English data they are trained on. Their performances on English tasks are often more robust than in other languages.
In this paper, we propose a method to empower the cross-lingual abilities of instructiontuned LLMs (It-LLMs) by building semantic alignment between languages. To achieve this, we introduce translation-following demonstrations to elicit better semantic alignment across languages. Our evaluations on multilingual question-answering benchmarks reveal that our models, tested in five distinct languages, outperform the performance of It-LLMs trained on monolingual datasets. The findings highlight the impact of translation-following demonstrations on non-English data, eliciting instructiontuning and empowering semantic alignment.