Lens: Rethinking Multilingual Enhancement for Large Language Models

ACL ARR 2025 February Submission5606 Authors

Abstract: As global demand for multilingual large language models (LLMs) grows, most LLMs still remain overly focused on English, leading to the limited access to advanced AI for non-English speakers. Current methods to enhance multilingual capabilities largely rely on data-driven post-training techniques, such as multilingual instruction tuning or continual pre-training. However, these approaches exhibit significant limitations, including high resource cost, exacerbation of off-target issue and catastrophic forgetting of central language abilities. To this end, we propose \textsc{Lens}, a novel approach that enhances multilingual capabilities by leveraging LLMs’ internal language representation spaces. \textsc{Lens} operates on two subspaces: the language-agnostic subspace, where it aligns target languages with the central language to inherit strong semantic representations, and the language-specific subspace, where it separates target and central languages to preserve linguistic specificity. Experiments on three English-centric LLMs show that \textsc{Lens} significantly improves multilingual performance while maintaining the model’s English proficiency, achieving better results with less computational cost compared to existing post-training approaches.

Paper Type: Long

Research Area: Special Theme (conference specific)

Research Area Keywords: Large Language Models, Multilingual Enhancement, Representation Engineering

Contribution Types: NLP engineering experiment

Languages Studied: English, Chinese, Japanese, Arabic, Korean, Bengali, Swahili, Spanish, French, German

Submission Number: 5606