Go to ACL ARR 2025 May homepageCOMI-LINGUA: Expert Annotated Large-Scale Dataset for Multitask NLP in Hindi-English Code-Mixing
Abstract: We introduce COMI-LINGUA, the largest manually annotated Hindi-English code-mixed dataset, comprising 125K+ high-quality instances across five core NLP tasks: Matrix Language Identification, Token-level Language Identification, POS Tagging, Named Entity Recognition, and Machine Translation. Each instance is annotated by three bilingual annotators, yielding over 376K expert annotations with strong inter-annotator agreement (Fleiss' Kappa >= 0.81). The rigorously preprocessed and filtered dataset covers both Devanagari and Roman scripts, and spans diverse domains such as social media text, news, and formal communications, ensuring real-world linguistic coverage. Evaluation reveals that closed-source LLMs significantly outperform traditional tools and open-source models. Notably, one-shot prompting consistently boosts performance across tasks, especially in structure-sensitive predictions like POS and NER, highlighting the effectiveness of prompt-based adaptation in code-mixed, low-resource settings. COMI-LINGUA is publicly available at: \url{https://anonymous.4open.science/r/CodeMixing/}.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: CodeMixing, Data Annotation,
Contribution Types: Data resources
Languages Studied: Hindi, English, Code-mixing
Submission Number: 6630
Loading