Volumetric Plane-based Rendering: A Novel Approach for Internal View Synthesis in 3D Gaussian Splatting

Sanghyuk Roy Choi, Minhyeok Lee

Keywords: 3D Gaussian Splatting, Medical Image, Novel View Synthesis, CT, MRI, Internal View Synthesis

TL;DR: GS-IR extends 3D Gaussian Splatting to internal cross-sectional reconstruction by selecting and optimizing plane-conditioned Gaussian subsets from volumetric medical data.

Abstract: 3D Gaussian Splatting (3DGS) represents scenes as sets of anisotropic 3D Gaussians optimized from multi-view exterior images. Training 3DGS is largely surface-driven and provides limited gradient for the volume interior. We investigate whether 3D Gaussians can capture internal volumetric structure when training is conditioned on cross-sectional slices. We propose GS-IR, which replaces exterior-view supervision with plane-conditioned internal rendering. GS-IR places virtual cameras along a semicircular trajectory and defines a thick cross-sectional plane passing through the volume center. A binary mask selects only the Gaussians whose centers fall within the plane; the selected subset is rasterized with the standard alpha-blending pipeline and optimized against the corresponding ground-truth slice obtained by trilinear interpolation of the input volume. Cycling through viewpoints provides supervision to Gaussians throughout the volume during training. This simple gradient-routing mechanism enables stable optimization of interior Gaussians and complements standard densification and pruning across existing 3DGS variants. We evaluate GS-IR on the KiTS23 (CT) and IXI (MRI) datasets. GS-IR can be applied to 3DGS, 2DGS, and Mip-Splatting without modifying their Gaussian rasterizers. It improves PSNR by up to +21.52 dB and SSIM by up to +0.54 over the respective baselines.

Submission Number: 17