Enhancing Modality-Agnostic Representations via Meta-learning for Brain Tumor Segmentation

Aishik Konwer

Abstract:

In medical vision, different imaging modalities provide
complementary information. However, in practice, not all
modalities may be available during inference or even train-
ing. Previous approaches, e.g., knowledge distillation or
image synthesis, often assume the availability of full modal-
ities for all subjects during training; this is unrealistic and
impractical due to the variability in data collection across
sites. We propose a novel approach to learn enhanced
modality-agnostic representations by employing a meta-
learning strategy in training, even when only limited full
modality samples are available. Meta-learning enhances
partial modality representations to full modality represen-
tations by meta-training on partial modality data and meta-
testing on limited full modality samples. Additionally, we
co-supervise this feature enrichment by introducing an aux-
iliary adversarial learning branch. More specifically, a
missing modality detector is used as a discriminator to
mimic the full modality setting. Our segmentation frame-
work significantly outperforms state-of-the-art brain tumor
segmentation techniques in missing modality scenarios.