CRANE: Causal Relevance Analysis of Language-Specific Neurons in Multilingual Large Language Models

ACL ARR 2026 January Submission4941 Authors

Keywords: Multilingual Large Language Models, Neuron-level Interpretability, Causal Intervention, Layer-wise Relevance Propagation, Language-specific Neurons, Mechanistic Interpretability, Model Analysis

Abstract: Multilingual large language models (LLMs) achieve strong performance across languages, yet how language capabilities are organized at the neuron level remains poorly understood. Prior work has identified language-related neurons mainly through activation-based heuristics, which conflate language preference with functional importance. We propose CRANE, a relevance-based analysis framework that redefines language specificity in terms of functional necessity, identifying language-specific neurons through targeted neuron-level interventions. CRANE characterizes neuron specialization by their contribution to language-conditioned predictions rather than activation magnitude. Our implementation will be made publicly available. Neuron-level interventions reveal a consistent asymmetric pattern: masking neurons relevant to a target language selectively degrades performance on that language while preserving performance on other languages to a substantial extent, indicating language-selective but non-exclusive neuron specializations. Experiments on English, Chinese, and Vietnamese across multiple benchmarks, together with a dedicated relevance-based metric and base-to-chat model transfer analysis, show that CRANE isolates language-specific components more precisely than activation-based methods.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: Multilingualism and Cross-Lingual NLP, Language Modeling, Interpretability and Analysis of Models for NLP, Machine Learning for NLP

Contribution Types: Model analysis & interpretability

Languages Studied: English, Chinese, Vietnamese

Submission Number: 4941