Training CNNs for Multimodal Glioma Segmentation with Missing MR Modalities

Karin van Garderen, Marion Smits, Stefan Klein

Keywords: convolutional neural network, glioma segmentation, multimodal

TL;DR: We adapted the well-known UNet architecture to produce CNNs that are robust to missing MR modalities in glioma segmentation.

Abstract: Missing data is a common problem in machine learning, and in retrospective imaging research it is often encountered in the form of missing imaging modalities. We propose to take into account missing modalities in the design and training of neural networks, to ensure that they are capable of providing the best possible prediction even when one of the modalities is not available. This would enable algorithms to be applied to subjects with fewer available modalities, without leaving out the same information in other subjects or applying data imputation. This concept is evaluated in the context of glioma segmentation, which is a problem that has received much attention in part due to the BraTS multi-modal segmentation challenge. The UNet architecture has been shown to be effective in this problem and therefore it serves as the reference method in this paper. To make the network robust to missing data we leveraged the dropout principle during training and applied this to the UNet architecture, but also to variations on the UNet architecture inspired by multimodal learning. These networks drastically improved the performance with missing modalities, while only performing slightly worse on the full dataset.

Code Of Conduct: I have read and accept the code of conduct.