Adapting BigScience Multilingual Model to Unseen LanguagesDownload PDF

09 Mar 2022, 15:07 (edited 10 Apr 2022)BigScience#5Readers: Everyone
  • Keywords: language adaptation, adapters
  • TL;DR: We explore ways to extend BigScience multilingual language model to two unseen languages (German and Korean).
  • Abstract: We benchmark different strategies of adding new languages (German and Korean) into the BigScience's pretrained multilingual language model with 1.3 billion parameters that currently supports 13 languages. We investigate the factors that affect the language adaptability of the model and the trade-offs between computational costs and expected performance.
1 Reply