LowREm: A Repository of Word Embeddings for 87 Low-Resource Languages Enhanced with Multilingual Graph Knowledge

ACL ARR 2024 June Submission1248 Authors

Abstract:

Contextualized embeddings based on large language models (LLMs) are available for various languages, but their coverage is often limited for lower resourced languages. Training LLMs for such languages is often difficult due to insufficient data and high computational cost. Especially for very low resource languages, static word embeddings thus still offer a viable alternative. There is, however, a notable lack of comprehensive repositories with such embeddings for diverse languages. To address this, we present LowREm, a centralized repository of static embeddings for 87 low-resource languages. We also propose a novel method to enhance GloVe-based embeddings by integrating multilingual graph knowledge, utilizing another source of knowledge, which is beneficial especially for low-resource languages.
We demonstrate the superior performance of our enhanced embeddings as compared to contextualized embeddings extracted from XLM-R on sentiment analysis. Our code and data are publicly available under URL.

Paper Type: Short

Research Area: Multilingualism and Cross-Lingual NLP

Research Area Keywords: less-resourced languages, multilingualism, resources for less-resourced languages, multilingual representations, endangered languages, indigenous languages

Contribution Types: NLP engineering experiment, Approaches to low-resource settings, Publicly available software and/or pre-trained models

Languages Studied: Swati, Sardinian, Yoruba, Guarani, Quechua, Northern Sotho, Limburgish, Lingala, Wolof, Zulu, Romansh, Igbo, Ganda, Assamese, Tswana, Haitian, Oromo, Sundanese, Bosnian, Breton, Scottish Gaelic, Xhosa, Malagasy, Javanese, Frisian, Sanskrit, Burmese, Uyghur, Yiddish, Oriya, Hausa, Lao, Sindhi, Tamil Romanized, Somali, Telugu Romanized, Kurdish, Punjabi, Pashto, Irish, Amharic, Urdu Romanized, Khmer, Uzbek, Bengali Romanized, Kyrgyz, Burmese (Zawgyi), Welsh, Gujarati, Esperanto, Afrikaans, Swahili, Marathi, Kannada, Nepali, Mongolian, Sinhala, Telugu, Latin, Belarussian, Tagalog, Macedonian, Galician, Armenian, Icelandic, Malayalam, Bengali, Urdu, Kazakh, Georgian, Azerbaijani, Albanian, Tamil, Estonian, Latvian, Malay, Slovenian, Lithuanian, Hebrew, Slovak, Greek, Thai, Bulgarian, Danish, Ukrainian, Romanian, Indonesian

Submission Number: 1248