NorBench -- A Benchmark for Norwegian Language Models

David Samuel, Andrey Kutuzov, Samia Touileb, Erik Velldal, Lilja Øvrelid, Egil Rønningstad, Elina Sigdel, Anna Sergeevna Palatkina

Keywords: language models, benchmarking, evaluation

TL;DR: Benchmark for Norwegian language models; new SOTA masked language models for Norwegian

Abstract: We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduce a range of new Norwegian language models (both encoder and encoder-decoder based). Finally, we compare and analyze their performance, along with other existing LMs, across the different benchmark tests of NorBench.

Student Paper: Yes, the first author is a student

Community Implementations: [ 8 code implementations](https://www.catalyzex.com/paper/arxiv:2305.03880/code)