Iterative Multilingual Spectral Attribute Erasure

ACL ARR 2025 February Submission7109 Authors

Abstract: Multilingual representations embed words with similar meanings to share a common semantic space across languages, creating opportunities for transferring debiasing effects between languages. However, existing methods typically operate on individual languages, show limited transferability of debiasing effects across languages. We present MUSAL (MUltilingual Spectral Attribute removaL), which identifies and mitigates joint bias subspaces across multiple languages through iterative SVD-based truncation. Evaluating MUSAL across eight languages and five demographic dimensions, we demonstrate its effectiveness in both standard and zero-shot settings, where target language data is unavailable but linguistically similar languages can be used for debiasing. Our comprehensive experiments across diverse language models (BERT, LLaMA, Mistral) show MUSAL outperforms traditional monolingual and cross-lingual approaches while maintaining model utility.

Paper Type: Long

Research Area: Ethics, Bias, and Fairness

Research Area Keywords: Multilingual NLP, Spectral Learning, Protected Attribute Removal, Iterative Methods

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Reproduction study, Approaches to low-resource settings, Approaches low compute settings-efficiency

Languages Studied: English, Spanish, French, Portuguese, German, Russian, Polish

Submission Number: 7109