Translation with context in LLMs: A study in Korean Idioms

ACL ARR 2025 February Submission8423 Authors

Abstract:

Despite advancements in Large Language Mod-001
els (LLMs), translation quality of generated002
outputs still remains inconsistent, particularly003
due to the misalignment in corresponding ex-004
pressions across source and target languages.005
In this paper, we study the behavior of LLMs,006
focusing on the translational strategies of non-007
compositional expressions or idiomatic expres-008
sions. While LLMs are capable of translating009
non-compositional expressions as shown by the010
high average COMET score of 0.7969, a high011
inconsistent corresponding idiomatic transla-012
tion accuracy of across multiple context sen-013
tences for the same idiom indicate a lack of014
deeper understanding of the idiom and its sur-015
rounding context. Our results provide a starting016
point to understand how LLMs process and017
handle non-compositional expressions.

Paper Type: Short

Research Area: Machine Translation

Research Area Keywords: evaluation; MT theory

Contribution Types: Model analysis & interpretability, Data analysis, Theory

Languages Studied: Korean, English

Submission Number: 8423