S2GO: Streaming Sparse Gaussian Occupancy

Jinhyung Park, Chensheng Peng, yihan hu, Wenzhao Zheng, Kris Kitani, Wei Zhan

Keywords: 3D Gaussian Splatting, 3D Occupancy Estimation, Autonomous Driving

Abstract: Despite the efficiency and performance of sparse query-based representations for detection, state-of-the-art 3D occupancy estimation methods still rely on voxel-based or dense Gaussian-based 3D representations. However, dense representations are slow, and they lack flexibility in capturing the temporal dynamics of driving scenes. Distinct from prior work, we instead summarize the scene into a compact set of 3D queries which are propagated through time in an online, streaming fashion. These queries are then decoded into semantic Gaussians at each timestep. We couple our framework with a denoising rendering objective to guide the queries and their constituent Gaussians in effectively capturing scene geometry. Due to its efficient, query-based representation, S2GO achieves state-of-the-art performance on the nuScenes and KITTI occupancy benchmarks, outperforming prior art (e.g., GaussianWorld) by 2.7 IoU with 4.5x faster inference.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 9713