Improving multilingual language models for the NER task

Anonymous

Abstract: In this paper we test methods to improve the quality of cross-lingual transfer for under-resourced languages by means of more efficient mapping between embedding spaces, which helps to improve alignment between token embeddings.We test the method in the Named Entity Recognition task for a range of Slavic languages.The results of our experiments demonstrate improvement up to 8\% of F1 measure in comparison to the XLM-RoBERTa few-shot baseline. Error analysis shows that our method especially helps for resolving ambiguous expressions, probably because the improved representation is able to take into account more information available in better resourced languages.

Paper Type: short