Beyond Model Accuracy: Identifying Hidden Underlying Issues in Chest X-ray Classification

Richard Wainwright, Danny Wang, Harrison Layton, Alina Bialkowski

Abstract: As deep learning model performance continues to advance in detecting and classifying disease, it is important to show that these models are trustworthy and reliable for use by medical professionals. However, reporting of model accuracy alone can conceal underlying issues with the utilised data and model training, which could lead to serious consequences in practical applications. In this paper, we investigate machine learning models for chest X-ray disease classification using the COVID-19 Radiography Database (CRD), COVIDx and ChestX-ray14 (CXR-14) datasets. Existing literature has identified issues with these datasets, including spurious correlations and incorrect ground truth labels in the data. Through the utilisation of model attention visualisation, uncertainty measures, and low-dimensionality data representations, we underscore a suite of techniques capable of detecting such issues. Our procedure offers a means to visualise data quality issues and the uncertainty of model predictions that extend beyond numerical values, helping to improve understanding of true performance and trust (or distrust) in machine learning models.