Harnessing Linguistic Dissimilarity for Language Generalization on Unseen Low-Resource Varieties

Jinju Kim, Haeji Jung, Youjeong Roh, Jong Hwan Ko, David R. Mortensen

Keywords: dialects and language varieties; less-resourced languages, endangered languages, indigenous languages, minoritized languages

TL;DR: Unlike most transfer learning that collapses differences for generalization, for low-resource varieties, modeling both invariant and specific structure better supports generalization by harnessing linguistic dissimilarity.

Abstract:

Low-resource language varieties used by specific groups remain neglected in the development of Multilingual Language Models. A great deal of cross-lingual research focuses on inter-lingual language transfer which strives to align allied varieties and minimize differences between them. However, for low-resource varieties, linguistic dissimilarity is also an important cue allowing generalization to unseen varieties. Unlike prior approaches, we propose a two-stage Language Generalization framework that focuses on capturing variety-specific cues while also exploiting rich overlap offered by high-resource source variety. First, we propose TOPPing, a source-selection method specifically designed for low-resource varieties. Second, we suggest a lightweight VAÇAÍ-Bowl architecture that learns variety-specific attributes with one branch while a parallel branch captures variety-invariant attributes using adversarial training.
We evaluate our framework on structural prediction tasks, which are among the few tasks available, as proxy for performance on other downstream tasks.
Using VAÇAÍ-Bowl with TOPPing yields an average 54.62% improvement in the dependency parsing task, which serves as a proxy for performance on other downstream tasks across 10 low-resource varieties.

Scope Confirmation: To the best of my judgment, this submission falls within the scope of CoNLL.

Primary Area Selection: Typology and Multilinguality

Secondary Area Selection: Computational Social Science and Sociolinguistics

Use Of Generative Artificial Intelligence Tools: Yes, for editing/proofreading the manuscript

Data Collection From Human Subjects: No

Submission Type: Archival: I certify that the submission has not been previously published, nor is the material in it under review by another journal or conference. Further, no material in it will be submitted for review at another conference or journal while under review by CoNLL 2026.

Submission Number: 218