Counterfactual Attention Intervention for Multiple Instance Learning

Imane Chraki, Pierre Marza, Stergios Christodoulidis, Maria Vakalopoulou

Keywords: Multiple Instance Learning, Attention, Whole Slide Images

Abstract:

Attention-based Multiple Instance Learning (MIL) has become a standard approach for whole slide images (WSI) analysis. Existing MIL models however tend to learn attention weights that exploit shortcuts. In this work, we propose Counterfactual Attention MIL (CAL-MIL), a causal extension of attention-based MIL that explicitly measures and optimizes the contribution of attention to slide-level predictions. CAL-MIL evaluates attention quality through counterfactual interventions, comparing factual predictions with predictions obtained after replacing attention weights with randomized counterfactual distributions. This additional supervisory signal encourages the model to rely on truly discriminative patches and reduces dependence on spurious correlations.
Across X histopathology classification benchmarks (BRCA, NSCLC, LUAD ...), CAL-MIL achieves performance comparable to strong MIL baselines while providing more causally meaningful attention behavior. Qualitative perturbation experiments show that dropping the top-attended patches leads to larger performance degradation in CAL-MIL compared to ABMIL, highlighting the importance of causal supervision for reliable and interpretable WSI-based prediction.

Primary Subject Area: Application: Histopathology

Secondary Subject Area: Causality

Registration Requirement: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 310