DeCo-Net: Robust Multimodal Brain Tumor Segmentation via Decoupled Complementary Knowledge Distillation

Zhilin Zeng, Zelin Peng, Xiaokang Yang, Wei Shen

Abstract: Automated brain tumor segmentation with multimodal magnetic resonance imaging (MRI) plays a pivotal rule in clinical application. However, most existing algorithms require complete image modalities as input, which is often impractical to obtain for every patient in real clinical practice. Therefore, a robust multimodal algorithm that is capable of handling various modality-incomplete data is highly desirable. In this paper, we propose DeCo-Net, a Decoupled Complementary knowledge distillation framework for multimodal brain tumor segmentation with incomplete modalities. Specifically, our approach decouples the feature learning of the modality-incomplete data into two branches: one dedicated to extracting the inherent features from the available modalities and the other focused on inferring the complementary missing modal information. We employ a teacher-student co-training framework where the teacher network is collaboratively trained to dynamically transfer the complementary knowledge to the student model based on the specific type of modality-incomplete data fed to student. To this end, we propose a modality-aware contrastive distillation strategy that guides the student model to distill a discriminative and complementary knowledge representation that acts as supplements to the original modality-incomplete representation. Extensive evaluations on the BraTS2018, BraTS2020 and BraTS2023 datasets demonstrate that our method achieves state-of-the-art performance in multimodal brain tumor segmentation with incomplete modalities.

External IDs:dblp:conf/bibm/ZengPY024