Learning Compact 3D Gaussians via Feed-Forward Point Fusion

Brandon Smart, Chuanxia Zheng, Iro Laina, Victor Adrian Prisacariu

Keywords: 3D Reconstruction, 3D Gaussian Splatting, Feed-Forward Reconstruction

TL;DR: We introduce Splatt3RFusion, a neural network that reconstructs compact and high-quality 3D Gaussians from unposed images in a single forward pass of a neural network, merging nearby primitives using an octree structure.

Abstract: We present Splatt3RFusion, a feed-forward neural network that reconstructs compact and high-quality 3D Gaussians directly from a set of unposed and uncalibrated images. Unlike prior feed-forward methods that typically predict one 3D Gaussian primitive per pixel in each image -- producing severe redundancy, duplication, and ghosting on one physical surface -- our approach efficiently fuses points in 3D space through a multi-scale octree structure, yielding a compact and coherent representation. Built upon VGGT, a foundation model for pose-free 3D geometry prediction, Splatt3RFusion introduces a Gaussian prediction branch that infers primitive parameters using only photometric supervision. We also introduce the ability to control the number of 3D Gaussians generated at test-time, allowing for a controllable tradeoff between PSNR and the number of 3D Gaussian primitives used. The model is efficient, reducing both memory usage and rendering cost, while achieving state-of-the-art results on RealEstate10k and ScanNet++.

Submission Number: 299