3D Fetal Pose Estimation with Adaptive Variance and Conditional Generative Adversarial Network

Junshen Xu, Molin Zhang, Esra Abaci Turk, Patricia Ellen Grant, Polina Golland, Elfar Adalsteinsson

Abstract: Fetal motion is the dominant challenge to reliable performance and diagnostic quality of fetal magnetic resonance imaging (MRI). The fetus can move unpredictably and rapidly, leading to severe image artifacts. Consequently, MR acquisitions are largely limited to so-called single-shot techniques in an attempt to “freeze” fetal motion through fast imaging, while the problem due to motion occur between slices still exists. In this work, we propose a deep learning method for fetal pose estimation from MR volumes using the paradigm of conditional generative adversarial network which consists of two networks, a generator and a discriminator. The generator is responsible for estimating keypoint heatmaps from input MRI and the discriminator tries to learn the features of plausible fetal pose and distinguish ground-truth heatmaps from generated ones. With this adversarial training scheme, the generator can robustly produce realistic heatmaps for fetal pose inference. Besides, we use adaptive variance to model the difference in intensity of motion of different keypoints. Evaluation shows that the proposed method can improve the performance of pose estimation in 3D MRI, achieving quantitatively an average error of 2.64 mm and 98.31% accuracy (with error less than 10 mm). The proposed method can process volumes with latency less than 300 ms, potentially enabling low-latency online tracking of fetal pose during MR scans.