Cross-Modality Neuroimage Synthesis: A Survey

Guoyang Xie, Jinbao Wang, Yawen Huang, Feng Zheng, Yefeng Zheng, Yaochu Jin

Abstract: Multi-modality imaging improves disease diagnosis and reveals distinct deviations in tissues with anatomical properties. The existence of completely aligned and paired multi-modality neuroimaging data has proved its effectiveness in brain research. However, collecting fully aligned and paired data is expensive or even impractical, since it faces many difficulties, including high cost, long time duration, image corruption, and privacy issues. An alternative solution is to explore either unsupervised or weakly supervised learning methods to synthesize the absent neuroimaging data. In this paper, we provide a comprehensive review of cross-modality synthesis for neuroimages, from the perspectives of weakly supervised and unsupervised settings, loss functions, evaluation metrics, ranges of modality, datasets, and the synthesis-based downstream applications. We begin with highlighting several opening challenges for cross-modality neuroimage synthesis. Then, we discuss representative architectures of cross-modality synthesis methods under different supervisions. This is followed by a stepwise in-depth analysis to evaluate how cross-modality neuroimage synthesis improves the performances of its downstream tasks. Finally, we summarize the existing research findings and point out future research directions. All resources are available at \href{https://github.com/M-3LAB/awesome-multimodal-brain-image-systhesis}{https://github.com/M-3LAB/awesome-multimodal-brain-image-systhesis}.