Towards Robust Zero-shot Chest X-ray Classification - Exploring Data Distribution Bias in Chest X-ray Datasets

Sheethal Bhat, Adarsh Bhandary Panambur, Awais Mansoor, Bogdan Georgescu, Sasa Grbic, Andreas K. Maier

Abstract: In recent years, unsupervised classification models have become increasingly significant, primarily due to the difficulties associated with data labeling and its costs. This trend is also notable in the field of medical imaging, particularly with chest X-rays (CXRs). Among the various unsupervised pretraining methodologies, image-text models like CLIP are highlighted for their considerable enhancements in zero-shot classification. In this study, we perform a detailed analysis of CLIP’s performance using multiple large CXR datasets, investigating how the batch size, dataset size, and distribution biases differentially influence outcomes across various findings. In two distinct experiments,we showan average of 3% enhancement in the macro average zero-shot AUC scores when the batch size is increased, and a corresponding 8% improvement for pneumothorax by the addition of a second dataset. For pleural effusion, where performance is nearly saturated and previous changes had little effect, we examine adding weak supervisory meta-labels and image-to-image contrastive loss, achieving an average 1% improvement in zero-shot AUC. Consequently, our work shows incorporating dataset insights, meta-information and contrastive learning strategies enhances the robustness and accuracy of CLIP-CXR for specific findings.

External IDs:dblp:conf/bildmed/BhatPMGGM25