Controlling Language Confusion in Multilingual LLMs

Nahyun Lee, Yeongseo Woo, Hyunwoo Ko, Guijin Son

Keywords: Language confusion, Multilingual

Abstract:

Large language models often suffer from language confusion, a phenomenon where responses are partially or entirely generated in unintended languages. This can critically impact user experience in low-resource settings. We hypothesize that conventional supervised fine-tuning exacerbates this issue because the softmax objective focuses probability mass only on the single correct token but does not explicitly penalize cross-lingual mixing.
Interestingly, by observing loss trajectories during the pretraining phase, we observe that models fail to learn to distinguish between monolingual and language-confused text. Additionally, we find that ORPO, which adds penalties for unwanted output styles to standard SFT, effectively suppresses language-confused generations even at high decoding temperatures without degrading overall model performance. Our findings suggest that incorporating appropriate penalty terms can mitigate language confusion in low-resource settings with limited data.

Archival Status: Archival

Paper Length: Short Paper (up to 4 pages of content)

Submission Number: 316