Learning a Metric without Supervision: Multimodal Registration using Synthetic Cycle Discrepancy

Hanna Siebert, Lasse Hansen, Mattias P Heinrich

Keywords: multimodal features, image registration, self-supervision

Abstract: Training deep learning based medical image registration methods involves the challenge of finding a suitable metric. To avoid the difficulty of choosing a metric for multimodal image registration, we propose a completely new concept relying on geometric instead of metric supervision with three-way registration cycles. Therefore, we create a synthetic image by applying a synthetic transformation on one of the input images. This leads to cycles that for each pair of input images comprise two multimodal transformations to be estimated and one known synthetic monomodal transformation. We minimise the discrepancy between the combined multimodal transformations and the synthetic monomodal transformation. By minimising this cycle discrepancy, we are able to learn multimodal registration between CT and MRI without metric supervision. Our method outperforms state-of-the-art metric supervision and comes very close to fully-supervised learning with ground truth labels.

Paper Type: methodological development

Primary Subject Area: Image Registration

Secondary Subject Area: Unsupervised Learning and Representation Learning

Paper Status: original work, not submitted yet

Source Code Url: https://github.com/multimodallearning/learning_without_metric

Data Set Url: https://github.com/multimodallearning/learning_without_metric

Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.

Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.