Improving Generalized Zero-Shot Learning for Multi-LabelChest X-ray Classification Using Knowledge Graphs

Chinmay Prabhakar, Anjany Sekuboyina, Johannes C. Paetzold, Hongwei Li, Tamaz Amiranashvili, Jens Kleesiek, bjoern menze

Keywords: Graph Neural Network, Generalized Zero-Shot Learning, UMLS, Explainable AI

TL;DR: Generalized Zero Shot Learning on chest x-ray images using a universal knowledge graph

Abstract: Generalized zero-shot learning (GZSL) aims to develop models that can reliably label classes not encountered during training, while maintaining a good performance on the seen ones. This becomes especially challenging in the realm of multi-label chest X-ray image classification, due to the presence of numerous unknown disease-types and the limited information inherent to x-ray images. In this work, we present a knowledge graph-based approach to GZSL. Our method directly injects the semantic relationships between seen and unseen disease classes by making use of the Unified Medical Language System (UMLS ). Specifically, we use the UMLS as a knowledge base and device a principled approach of parsing and processing it, conditioned on the task at hand. We show that our method matches the labelling performance of the state-of-the-art while outperforming it on unseen classes (AU-ROC 0.68 vs. 0.66). We also demonstrate that embedding the disease-specific knowledge as a graph provides inherent explainability, which allows us to understand the multi-label relation and model decision

Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.

Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.

Paper Type: both

Primary Subject Area: Detection and Diagnosis

Secondary Subject Area: Application: Radiology

Confidentiality And Author Instructions: I read the call for papers and author instructions. I acknowledge that exceeding the page limit and/or altering the latex template can result in desk rejection.