Confidence-Calibrated Clinical Decision Support System for Reliable Respiratory Disease Screening

Micky C. Nnamdi, J. Ben Tamo, Wenqi Shi, Vivek Chundru, Benoit Louis Marteau, Oankar R. Patil, May Dongmei Wang

Keywords: respiratory disease, clinical decision support, audio signal analysis, confidence calibration

Abstract: With the growing adoption of computer-aided diagnostic and treatment recommendation systems in healthcare, it is essential to ensure both the accuracy and reliability of AI-enabled clinical decision support systems. In this study, we comprehensively examine existing model confidence calibration methods and propose an ensemble-based calibration approach for reliable predictions in clinical decision support systems (CDSSs). Specifically, we introduce an ENsemble-based Confidence-caLibrated deep neural network, ENCL-DNN, to improve respiratory disease screening using cough sounds. We also leverage local interpretable model-agnostic explanations to monitor the behavior of the CDSS, identifying the key features that contribute to its predictions and ensuring transparency in the diagnosis. By employing the ensemble-based calibration method, ENCL-DNN demonstrates superior performance on two publicly available respiratory audio datasets, Coswara and Cambridge, as evidenced by a 50% and a 28.74% reduction in Expected Calibration Error (ECE), respectively, compared to the uncalibrated baselines. Our experiments highlight the significance of well-calibrated deep neural networks in respiratory disease screening and the enhancement of reliability in mobile healthcare systems. By providing reliable and transparent predictions, ENCL-DNN has the potential to promote the wide adoption of AI-driven CDSSs and thereby improve patient outcomes through early diagnosis and intervention.

Track: 4. AI-based clinical decision support systems

Registration Id: 2QNGBSMWDX2

Submission Number: 275