3D Transformer-GAN for High-Quality PET Reconstruction

Yanmei Luo, Yan Wang, Chen Zu, Bo Zhan, Xi Wu, Jiliu Zhou, Dinggang Shen, Luping Zhou

Abstract: To obtain high-quality positron emission tomography (PET) image at low dose, this study proposes an end-to-end 3D generative adversarial network embedded with transformer, namely Transformer-GAN, to reconstruct the standard-dose PET (SPET) image from the corresponding low-dose PET (LPET) image. Specifically, considering the convolutional neural network (CNN) can well describe the local spatial features, while the transformer is good at capturing the long-range semantic information due to its global information extraction ability, our generator network takes advantages of both CNN and transformer, and is designed as an architecture of EncoderCNN-Transformer-DecoderCNN. Particularly, the EncoderCNN aims to extract compact feature representations with rich spatial information by using CNN, while the Transformer targets at capturing the long-range dependencies between the features learned by the EncoderCNN. Finally, the DecoderCNN is responsible for restoring the reconstructed PET image. Moreover, to ensure the similarity of voxel-level intensities as well as the data distributions between the reconstructed image and the real image, we harness both the voxel-wise estimation error and the adversarial loss to train the generator network. Validations on the clinical PET data show that our proposed method outperforms the state-of-the-art methods in both qualitative and quantitative measures.