Keywords: Deep learning, linear conditioning, segmentation, metadata, task adaptation.
TL;DR: This work adapts a linear conditioning method for image segmentation models enabling integration of metadata and multi-class training with few or missing labels.
Abstract: Medical images are often accompanied by metadata describing the image (vendor, acquisition parameters) and the patient (disease type or severity, demographics, genomics). This metadata is usually disregarded by image segmentation methods. In this work, we adapt a linear conditioning method called FiLM (Feature-wise Linear Modulation) for image segmentation tasks. This FiLM adaptation enables integrating metadata into segmentation models for better performance. We observed an average Dice score increase of 5.1% on spinal cord tumor segmentation when incorporating the tumor type with FiLM. The metadata modulates the segmentation process through low-cost affine transformations applied on feature maps which can be included in any neural network's architecture. Additionally, we assess the relevance of segmentation FiLM layers for tackling common challenges in medical imaging: training with limited or unbalanced number of annotated data, multi-class training with missing segmentations, and model adaptation to multiple tasks. Our results demonstrated the following benefits of FiLM for segmentation: FiLMed U-Net was robust to missing labels and reached higher Dice scores with few labels (up to 16.7%) compared to single-task U-Net. The code is open-source and available at www.ivadomed.org.
Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.
Source Code Url: https://github.com/ivadomed/ivadomed
Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.
Data Set Url: https://kits19.grand-challenge.org/, http://medicaldecathlon.com/
Paper Type: methodological development
Source Latex: zip
Primary Subject Area: Segmentation
Secondary Subject Area: Detection and Diagnosis