Conformal Prediction for ECG Interpretation: A Study on Human-AI Collaboration in Clinical Decision Support

Duarte Folgado, Lorenzo Famiglini, Andrea Campagner, Hélder Dores, Marília Barandas, Hugo Gamboa, Federico Cabitza

Abstract: Despite the recognized potential of Artificial Intelligence (AI)-based decision support systems for electrocardiogram (ECG) interpretation, the complex interactions between AI-generated advice and its presentation to physicians through user interfaces still need to be comprehensively understood. Clinical ECG interpretation, like many other healthcare scenarios, is challenging due to overlapping findings across different conditions, leading to inherent uncertainty that physicians must navigate. Despite this, AI-based systems typically present a single option as their advice. In contrast, set-valued support, which predicts a set of possible classes rather than a single outcome, may offer a more natural approach to addressing clinical uncertainty. In this paper, we report a comparative study investigating the impact of single-valued versus set-valued support systems on the accuracy of ECG interpretation. We conducted a Wizard-of-Oz study involving 62 cardiologists, divided into two groups receiving either single- or set-valued support, with an additional layer of textual explanations of the AI advice. Our results reveal that set-valued support significantly improved diagnostic accuracy, particularly in complex cases (Cohen’s d = .53) and for cardiologists with less than 10 years of experience (Cohen’s d = .67). Including case-pertinent textual explanations further enhanced the diagnostic accuracy (Cohen’s d = .62). These results highlight the potential of set-valued support in medical diagnostics, especially for clinical cases of high complexity. Set-valued support promotes human integration and control, offering a pathway for more robust and human-centered AI-assisted healthcare solutions.