Mixed Prototype Correction for Causal Inference in Medical Image Classification

Yajie Zhang, Zhi-An Huang, Zhiliang Hong, Songsong Wu, Jibin Wu, KC Tan

Abstract:

The heterogeneity of medical images poses significant challenges to accurate disease diagnosis. To tackle this issue, the impact of such heterogeneity on the causal relationship between image features and diagnostic labels should be incorporated into model design, which however remains underexplored. In this paper, we propose a mixed prototype correction for causal inference (MPCCI) method, aimed at mitigating the impact of unseen confounding factors on the causal relationships between medical images and disease labels, so as to enhance the diagnostic accuracy of deep learning models. The MPCCI comprises a causal inference component based on front-door adjustment and an adaptive training strategy. The causal inference component employs a multi-view feature extraction (MVFE) module to establish mediators, and a mixed prototype correction (MPC) module to execute causal interventions. Moreover, the adaptive training strategy incorporates both information purity and maturity metrics to maintain stable model training. Experimental evaluations on four medical image datasets, encompassing CT and ultrasound modalities, demonstrate the superior diagnostic accuracy and reliability of the proposed MPCCI. The code will be available at https://github.com/Yajie-Zhang/MPCCI.

Primary Subject Area: [Content] Vision and Language

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: The work presented introduces a novel method, Mixed Prototype Correction for Causal Inference (MPCCI), which significantly contributes to multimedia/multimodal processing, particularly in the domain of medical image modality. MPCCI addresses the challenges posed by the heterogeneity of medical images on accurate disease diagnosis, a crucial aspect of multimodal processing. By incorporating causal inference techniques, MPCCI aims to understand and mitigate the impact of unseen confounding factors on the causal relationships between medical images and disease labels.

Submission Number: 3737