Causal Attribution of Model Performance Gaps in Medical Imaging Under Distribution Shifts

Pedro M Gordaliza, Nataliia Molchanova, Jaume Banus, Meritxell Bach Cuadra

Keywords: Gap, Shift, Segmentation, Annotations, Causality

Abstract: Deep learning models for medical image suffer significant performance drops due to distribution shifts, but the causal mechanisms behind these drops remain poorly understood. We extend causal attribution frameworks to high-dimensional problems, quantifying how acquisition protocols and annotation variability contribute to performance degradation. We model the data-generating process through a causal graph and employ Shapley symmetry to fairly attribute performance changes to individual mechanisms. Our framework addresses unique challenges in medical imaging: high-dimensional input, limited samples, and complex mechanism interactions. Validation in multiple sclerosis (MS) lesion segmentation reveals that annotation protocol shifts account for X0.4\% ($\pm$10.6\%) of Dice drop when models encounter new annotators, while acquisition shifts dominate (X0.4\% $\pm$2.0\%) when crossing imaging centers. This actionable insight enables practitioners to prioritize targeted interventions. Our code and is available at Anon. repository

Submission Number: 28