Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation

Zhenkai Zhang, Markus Hiller, Krista A. Ehinger, Tom Drummond

Keywords: Medical Imaging, 3D Diffusion Model, Diffusion Transformer, CT Scan, Medical Image Generation

TL;DR: We introduce 'PRDiT', a scalable generative model that synthesizes high-quality 3D CT volumes by using a two-stage approach that enhances training stability and preserves fine details, and outperforms existing models on key medical imaging datasets.

Abstract: Generating high-resolution 3D CT volumes with fine details remains challenging due to substantial computational demands and optimization difficulties inherent to existing generative models. In this paper, we propose the Pixel-Level Residual Diffusion Transformer (PRDiT), a scalable generative framework that synthesizes high-quality 3D medical volumes directly at voxel-level. PRDiT introduces a two-stage training architecture comprising 1) a local denoiser in the form of an MLP-based blind estimator operating on overlapping 3D patches to separate low-frequency structures efficiently, and 2) a global residual diffusion transformer employing memory-efficient attention to model and refine high-frequency residuals across entire volumes. This coarse-to-fine modeling strategy simplifies optimization, enhances training stability, and effectively preserves subtle structures without the limitations of an autoencoder bottleneck. Extensive experiments conducted on the LIDC-IDRI and RAD-ChestCT datasets demonstrate that PRDiT consistently outperforms state-of-the-art models, such as HA-GAN, 3D LDM and WDM-3D, achieving significantly lower 3D FID, MMD and Wasserstein distance scores.

Primary Area: generative models

Submission Number: 8201