Synergistic PET/MR Reconstruction with VAE Constraint

Valentin Gautier, Claude Comtat, Florent Sureau, Alexandre Bousse, Voichita Maxim, Bruno Sixou

Abstract: In this work, we propose a synergistic PET/MR reconstruction method based on the ADMM algorithm and a pre-trained bimodal Variational Auto Encoder (VAE) as a constraint. Because of the multiple modalities, balancing the VAE's loss becomes a challenge. To solve this, we adapt an adaptive loss balancing method and apply it to the training of the VAE. We evaluate our approach on 2D slices from 44 different patients and show that the presented approach performs particularly well on low dose/highly undersampled data.