Adapting BigScience Multilingual Model to Unseen LanguagesDownload PDF

Zheng Xin Yong, Vassilina Nikoulina

Published: 09 Apr 2022, Last Modified: 05 May 2023BigScience#5Readers: Everyone
Keywords: language adaptation, adapters
TL;DR: We explore ways to extend BigScience multilingual language model to two unseen languages (German and Korean).
Abstract: We benchmark different strategies of adding new languages (German and Korean) into the BigScience's pretrained multilingual language model with 1.3 billion parameters that currently supports 13 languages. We investigate the factors that affect the language adaptability of the model and the trade-offs between computational costs and expected performance.
1 Reply