Improving multi-modal brain tumor segmentation via pre-training and knowledge distillation based post-training

Weide Liu, Jingwen Hou, Xiaoyang Zhong, Huijing Zhan, Jun Cheng, Yuming Fang, Guanghui Yue

Abstract: Deep learning-based brain tumor segmentation (BTS) models for multi-modal MRI images have seen significant advancements in recent years. However, a common problem in practice is the unavailability of some modalities due to varying scanning protocols and patient conditions, making segmentation from incomplete MRI modalities a challenging issue. Previous methods have attempted to address this by fusing accessible multi-modal features, leveraging attention mechanisms, and synthesizing missing modalities using generative models. However, these methods ignore the intrinsic problems of medical image segmentation, such as the limited availability of training samples, particularly for cases with tumors. Furthermore, these methods require training and deploying a specific model for each subset of missing modalities. To address these issues, we propose a novel approach that enhances the BTS model from two perspectives. Firstly, we introduce a pre-training stage that generates a diverse pre-training dataset covering a wide range of different combinations of tumor shapes and brain anatomy. Secondly, we propose a post-training stage that enables the model to reconstruct missing modalities in the prediction results when only partial modalities are available. To achieve the pre-training stage, we conceptually decouple the MRI image into two parts: ‘anatomy’ and ‘tumor’. We pre-train the BTS model using synthesized data generated from the anatomy and tumor parts across different training samples. For the post-training stage, we introduce a knowledge distillation-based process that enables the model to adapt to partial-modality inputs. This process removes one modality intentionally from the input and encourages the model to produce outputs identical to those when all modalities are present. This allows the model to reconstruct missing information through the post-training process. Extensive experiments demonstrate that our proposed method significantly improves the performance over the baseline and achieves new state-of-the-art results on three brain tumor segmentation datasets: BRATS2020, BRATS2018, and BRATS2015. The code is available at https://github.com/ZhongAobo/Asymmetry-BTS.

External IDs:dblp:journals/ijon/LiuHZZCFY25