Comparative Study of Named Entity Recognition Models

ACL ARR 2024 June Submission2125 Authors

Abstract: Named Entity Recognition (NER) is a fundamental and non-trivial task in natural language processing, that is crucial for various downstream applications. This paper presents a comprehensive comparative study of NER performance across a spectrum of state-of-the-art models, with a particular focus on the adaptation and fine-tuning of Question Answering (QA) models, such as BERT and RoBERTa, alongside prominent text generation models, including Llama2 (Touvron et al., 2023), Mistral (Jiang et al., 2023), and ChatGPT3.5-Turbo. In this study, we explore the efficacy of QA models when repurposed and adapted to NER tasks and additionally, we examine the zero-shot capabilities of Large Language Models, utilizing them without task-specific fine-tuning to assess their innate ability to recognize named entities. Through extensive experimentation on the benchmark dataset BUSTER, we analyze and compare the precision, recall, and F1 scores of each model variant across various NER categories. Furthermore, we investigate the robustness of these models under different training regimes and evaluation metrics

Paper Type: Long

Research Area: Information Extraction

Research Area Keywords: named entity recognition and relation extraction, document-level extraction, zero/few-shot extraction

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 2125