Rendering 3D CT Scans through 3D Gaussian Splatting Initialized with Points Sampled by Cube-Based Neural Radiance Fields

Sanghyuk Roy Choi, Chanhoe Gu, Sun Jae Baek, Minhyeok Lee

Abstract: Inthis paper, we present Cube-Cloud 3D Gaussian Splatting (Cube-Cloud 3DGS), a novel framework designed to render medical image that inherently structured with three fixed axes. The methods such as COLMAP, Neural Radiance Field (NeRF), and 3D Gaussian Splatting are unsuitable for medical image reconstruction due to the lack of diverse viewpoints. To address this challenge, we propose Cube-Cloud 3DGS that leverages Cube-based Neural Radiance Field (CuNeRF) for cube-based sampling to generate point clouds from medical data. CuNeRF in Cube-Cloud 3DGS generates point cloud and renders images through various viewpoints which can be used as camera poses. We integrate the point cloud with 3D Gaussian Splatting that is initializing 3D gaussians. By utilizing the viewpoints extracted from CuNeRF, the parameters of 3D gaussians are refined. Cube-Cloud 3DGS renders images through its 3D gaussians while traditional models fail to render based on medical images. We evaluated Cube-Cloud 3DGS on the Kidney and Kidney Tumor Segmentation (KiTS23) dataset, demonstrating that our model reconstructs 3D medical volumes effectively. Therefore, our model resolves the limitation focusing on the internal features for medical images. Our model achieves higher performance of 2.707 in PSNR and 0.0504 in SSIM over existing 3D Gaussian Splatting.

External IDs:dblp:conf/icaiic/ChoiGBL25