MR to CT Synthesis Using 3d Latent Diffusion

Austin Tapp, Abhijeet Parida, Can Zhao, Van Lam, Natasha Leporé, Syed Muhammad Anwar, Marius George Linguraru

Abstract: Diffusion probabilistic models are recognized for generating realistically appearing synthetic images, but producing 3D medical images remains computationally intensive. Further, latent diffusion for synthesizing medical volumes has focused on generating images of the same modality as training data. This study introduces cross-modality synthesis using 3D latent diffusion for generating computed tomography (CT) volumes from magnetic resonance imaging (MRI). We train an autoencoder to reconstruct CT via denoising diffusion probabilistic models using a novel MRI-CT latent space. The image generation method is formulated so that the user may produce synthetic CT (sCT), preserving anatomical features, or further noise the latent space, to generate CTs with similar but unique anatomical features, all without model retraining or tuning. Evaluation on public adult and private pediatric datasets demonstrate Fréchet inception distance of 3.28 and perceptual similarity of 0.28. Skull segmentations from sCTs have 72% dice similarity compared to segmentations from true CTs. Our method is the first to demonstrate lightweight, cross-modality synthesis for 3D medical images that includes optional divergence from input anatomical features through latent diffusion.