Quantile Rendering: Efficiently Embedding High-dimensional Feature on 3D Gaussian Splatting

Yoonwoo Jeong, Cheng Sun, Yu-Chiang Frank Wang, Minsu Cho, Jaesung Choe

Keywords: Gaussian Splatting

TL;DR: Efficient rendering algorithm for high-dimensional feature rendering from 3D Gaussians.

Abstract:

Recent advancements in computer vision have successfully extended Open-vocabulary segmentation (OVS) to the 3D domain by leveraging 3D Gaussian Splatting (3D-GS).
Despite this progress, efficiently rendering the high-dimensional features required for open-vocabulary queries poses a significant challenge.
Existing methods employ codebooks or feature compression, causing information loss, thereby degrading segmentation quality.
To address this limitation, we introduce Quantile Rendering (Q-Render), a novel rendering strategy for 3D Gaussians that efficiently handles high-dimensional features while maintaining high fidelity.
Unlike conventional volume rendering, which densely samples all 3D Gaussians intersecting each ray, Q-Render sparsely samples only those with dominant influence along the ray.
By integrating Q-Render into a generalizable 3D neural network, we also propose Gaussian Splatting Network (GS-Net), which predicts Gaussian features in a generalizable manner.
Extensive experiments on ScanNet and LeRF demonstrate that our framework outperforms state-of-the-art methods, while enabling real-time rendering with an approximate ${\sim}43.7\times$ speedup on 512-D feature maps.
Code will be made publicly available.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 7824