What Matters When Building Universal Multilingual Named Entity Recognition Models?

ACL ARR 2026 January Submission5485 Authors

Keywords: multilingual named entity recognition

Abstract: Recent progress in universal multilingual named entity recognition (NER) has been driven by advances in multilingual transformer models and task-specific architectures, loss functions, and training datasets. Despite substantial prior work, we find that many critical design decisions for such models are made without systematic justification, with architectural components, training objectives, and data sources evaluated only in combination rather than in isolation. We argue that these decisions impede progress in the field by making it difficult to identify which choices improve model performance. In this work, we conduct extensive experiments around architectures, transformer backbones, training objectives, and data composition across a wide range of languages. Based on these insights, we introduce Otter, a universal multilingual NER model supporting over 100 languages. Otter achieves consistent improvements over strong multilingual NER baselines, outperforming GLiNER-x-base by 5.3pp in F1 and achieves competitive performance compared to large generative models such as Qwen3-32B, while being substantially more efficient. We release model checkpoints, training and evaluation code to facilitate reproducibility and future research.

Paper Type: Long

Research Area: Information Extraction and Retrieval

Research Area Keywords: multilingual open information extraction

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: 100+ languages (please refer to XTERME and XLM-R for full overview of languages)

Submission Number: 5485