Assessing European and Brazilian Portuguese LLMs for NER in Specialised Domains

Rafael Oleques Nunes, Joaquim Santos, Andre Suslik Spritzer, Dennis Giovani Balreira, Carla Maria Dal Sasso Freitas, Fernanda Olival, Helena Freire Cameron, Renata Vieira

Abstract: This paper discusses the impact of Portuguese variants in Large Language Models for the task of named entity recognition (NER) in specialised domains. The tests were made on a Brazilian Portuguese legal and a European Portuguese historical corpora. The models taken into account are BERTimbau (PT-BR), Albertina (PT-PT and PT-BR), and XML-R (multilingual). The impact was more evident in the Portuguese historical corpus, which resulted in higher F1 measures compared to previous works that did not consider the same language variant. Additionally, the study underscores the impact of model architecture on performance, highlighting the critical role of both linguistic alignment and model size in enhancing NER in specialised domains.