XTransplant: A Probe into the Upper Bound Performance of Multilingual Capability and Culture Adaptability in LLMs via Mutual Cross-lingual Feed-forward Transplantation

ACL ARR 2025 February Submission3768 Authors

Abstract:

Current large language models (LLMs) often exhibit imbalances in multilingual capabilities and cultural adaptability, largely attributed to their English-centric pretraining data.
To address this imbalance, we propose $\mathcal{X}$Transplant, a novel probing method that investigates cross-lingual latent interactions through innovative cross-lingual feed-forward transplantation during inference.
This approach enables models to leverage the strengths of both English and non-English languages.
Through extensive pilot studies, we empirically demonstrate the significant potential of $\mathcal{X}$Transplant in improving both the multilingual capabilities and cultural adaptability of LLMs, respectively from $\texttt{En} \rightarrow \texttt{non-En}$ and $\texttt{non-En} \rightarrow \texttt{En}$, highlighting the underutilization of current LLMs' multilingual potential.
Building on these insights, we develop an offline scaling inference strategy that achieves consistent performance improvements in multilingual and culture-aware tasks, sometimes even surpassing multilingual supervised fine-tuning.
This work advances our understanding of cross-lingual latent interactions in LLMs while offering a practical, training-free solution for enhancing multilingual performance and cultural adaptability.

Paper Type: Long

Research Area: Multilingualism and Cross-Lingual NLP

Research Area Keywords: multilingualism, cross-lingual transfer, multilingual representations, culture adaptability

Contribution Types: NLP engineering experiment, Approaches to low-resource settings

Languages Studied: Arabic, Bulgarian, German, Greek, English, Spanish, French, Hindi, Russian, Swahili, Thai, Turkish, Urdu, Vietnamese, Chinese, Romanian, Estonian, Haitian Creole, Indonesian, Italian, Tamil

Submission Number: 3768