Multi-Label Generalized Zero Shot Chest Xray Classification Using Feature Disentanglement and Multi-Modal Dictionaries

Dwarikanath Mahapatra

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Generalized zero shot learning, multi-label, multi-modal, dictionary, feature disentanglement

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Generalized zero shot learning (GZSL) aims to correctly predict seen and unseen classes, and most GZSL methods focus on the single label case. However, medical images can have multiple labels as in the case of chest x-rays. We propose a novel multi-modal multi-label GZSL approach that leverages feature disentanglement and multi-modal dictionaries to synthesize features of unseen classes. Feature disentanglement extracts class specific features, which are used with text embeddings to learn a multi-modal dictionary. A subsequent clustering step identifies class centroids, all of which contribute to better multi-label feature synthesis. Compared to existing methods, our approach does not require class attribute vectors, which are an essential part of GZSL methods for natural images but are not available for medical images. Our approach outperforms state of the art GZSL methods for chest x-rays. We also analyse the performance of different loss terms in ablation studies.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5175