Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering

Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Van Nguyen Nguyen, Terrence Chen, Ziyan Wu

Keywords: 3DGS, 6DGS, CT, Volumetric Rendering

Abstract: Current neural volumetric rendering methods like NeRF and 3D Gaussian Splatting (3DGS) achieve photorealistic quality but require prohibitive per-scan optimization (30+ minutes for 3DGS, 10+ hours for NeRF), limiting clinical applicability. We propose Render-FM, a feedforward model that directly regresses 6D Gaussian Splatting parameters from CT volumes in a single 2.8-second forward pass—a 500× speedup. Our key innovation, Anatomy-Guided Priming (AGP), leverages segmentation masks and transfer functions to provide anatomically-informed initialization. Trained on 991 diverse CT scans, Render-FM employs a 3D U-Net architecture to predict per-voxel 6DGS parameters, enabling immediate real-time rendering (328+ FPS). Experiments demonstrate that Render-FM achieves superior quality compared to optimized baselines (27.30 vs 26.63 dB PSNR), with optional 89-second fine-tuning reaching 31.67 dB PSNR. Unlike per-scan methods, Render-FM generalizes to unseen anatomies, novel transfer functions, and compositional organ visualization without retraining. This advancement transforms clinical volumetric visualization, reducing preparation time from hours to seconds while maintaining or exceeding state-of-the-art quality.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 5360