BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning

ACL ARR 2025 February Submission4551 Authors

Abstract: This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE), spanning 53 languages and three KE datasets: zsRE, CounterFact, and WikiFactDiff. Cross-lingual KE, which requires knowledge edited in one language to generalize across diverse languages while preserving unrelated knowledge, remains underexplored. To address this, we systematically evaluate IKE under zero-shot, one-shot, and few-shot setups, including tailored metric-specific demonstrations. Our findings reveal that model scale and demonstration alignment critically govern cross-lingual editing efficacy, with larger models and tailored demonstrations significantly improving performance. Linguistic properties, particularly script type, strongly influence outcomes, with non-Latin languages underperforming due to issues like language confusion.

Paper Type: Long

Research Area: Multilingualism and Cross-Lingual NLP

Research Area Keywords: multilingual benchmark, multilingual evaluation, cross-lingual transfer

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data resources, Data analysis

Languages Studied: Afrikaans,Arabic,Azerbaijani,Belarusian,Bulgarian,Bengali,Catalan,Cebuano,Czech,Welsh,Danish,German,Greek,Spanish,Estonian,Basque,Persian,Finnish,French,Irish,Galician,Hebrew,Hindi,Croatian,Hungarian,Armenian,Indonesian,Italian,Japanese,Georgian,Korean,Latin,Lithuanian,Latvian,Malay,Dutch,Polish,Portuguese,Romanian,Russian,Slovak,Slovenian,Albanian,Serbian,Swedish,Tamil,Thai,Turkish,Vietnamese,Ukrainian,Urdu,Chinese

Submission Number: 4551