Multichannel input pixelwise regression 3D U-Nets for medical image estimation with 3 applications in brain MRI

Jueqi Wang, Derek Berger, David Mattie, Jacob Levman

Keywords: 3D U-Net, Image Synthesis, Medical Image Reconstruction, Deep Learning

Abstract: The U-Net is a robust general-purpose deep learning architecture designed for semantic segmentation of medical images, and has been extended to 3D for volumetric applications such as magnetic resonance imaging (MRI) of the human brain. An adaptation of the U-Net to output pixelwise regression values, instead of class labels, based on multichannel input data, has been developed in the remote sensing satellite imaging research domain. The pixelwise regression U-Net has only received limited consideration as a deep learning architecture in medical imaging for the image estimation/synthesis problem, and the limited work so far did not consider the application of 3D multichannel inputs. In this paper, we propose the use of the multichannel input pixelwise regression 3D U-Net (rUNet) for estimation of medical images. Our findings demonstrate that this approach is robust and versatile and can be applied to predicting a pending MRI examination of patients with Alzheimer's disease based on previous rounds of imaging, can perform medical image reconstruction (parametric mapping) in diffusion MRI, and can be applied to the estimation of one type of MRI examination from a collection of other types. Results demonstrate that the rUNet represents a single deep learning architecture capable of solving a variety of image estimation problems. Public domain code is provided.

Paper Type: both

Primary Subject Area: Image Synthesis

Secondary Subject Area: Application: Radiology

Paper Status: original work, not submitted yet

Source Code Url: https://github.com/stfxecutables/Multichannel-input-pixelwise-regression-u-nets

Data Set Url: https://www.humanconnectome.org/, http://adni.loni.usc.edu/, https://www.med.upenn.edu/cbica/brats2020/

Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.

Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.