LangSAE Editing: Improving Multilingual Information Retrieval via Post-hoc Language Identity Removal

ACL ARR 2026 January Submission6243 Authors

Keywords: Multilingual Information Retrieval, Dense Retrieval, Multilingual Sentence Embeddings, Sparse Autoencoders

Abstract: Dense retrieval in multilingual settings often searches over mixed-language collections, yet multilingual embeddings encode language identity alongside semantics. This language signal can inflate similarity for same-language pairs and crowd out relevant evidence written in other languages. We propose **LANGSAE EDITING**, a post-hoc sparse autoencoder trained on pooled embeddings that enables controllable removal of language-identity signal directly in vector space. The method identifies language-associated latent units using cross-language activation statistics, suppresses these units at inference time, and reconstructs embeddings in the original dimensionality, making it compatible with existing vector databases without retraining the base encoder or re-encoding raw text. Experiments across multiple languages show consistent improvements in ranking quality and cross-language coverage, with especially strong gains for script-distinct languages.

Paper Type: Long

Research Area: Information Extraction and Retrieval

Research Area Keywords: Information Retrieval and Text Mining, Multilingualism and Cross-Lingual NLP, Interpretability and Analysis of Models for NLP, Ethics Bias and Fairness

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: Arabic, Chinese, English, French, Hindi, Italian, Japanese, Portuguese, Russian, Spanish

Submission Number: 6243