A Comprehensive Evaluation of Biomedical Entity Linking Models

David Kartchner, Jennifer Deng, Shubham Lohiya, Tejasri Kopparthi, Prasanth Bathala, Daniel Domingo-Fernández, Cassie S. Mitchell

Submission Type: Regular Long Paper

Submission Track: Information Extraction

Submission Track 2: Information Retrieval and Text Mining

Keywords: Entity Linking, Entity Normalization, Candidate Generation, Biomedical Natural Language Processing

TL;DR: An evaluation on the robustness and errors of biomedical entity linking models across a broad range of data and entity types.

Abstract: Biomedical entity linking (BioEL) is the process of connecting entities referenced in documents to entries in biomedical databases such as the Unified Medical Language System (UMLS) or Medical Subject Headings (MeSH). The study objective was to comprehensively evaluate nine recent state-of-the-art biomedical entity linking models under a unified framework. We compare these models along axes of (1) accuracy, (2) speed, (3) ease of use, (4) generalization, and (5) adaptability to new ontologies and datasets. We additionally quantify the impact of various preprocessing choices such as abbreviation detection. Systematic evaluation reveals several notable gaps in current methods. In particular, current methods struggle to correctly link genes and proteins and often have difficulty effectively incorporating context into linking decisions. To expedite future development and baseline testing, we release our unified evaluation framework and all included models on GitHub at https://github.com/davidkartchner/biomedical-entity-linking

Submission Number: 2162