ReSplat: Learning Recurrent Gaussian Splats

Haofei Xu, Daniel Barath, Andreas Geiger, Marc Pollefeys

Keywords: Gaussian Splatting, Feed-Forward, Learning to Optimize, View Synthesis

TL;DR: A feed-forward recurrent Gaussian splatting model for iteratively refining 3D Gaussians

Abstract:

While feed-forward Gaussian splatting models offer computational efficiency and can generalize to sparse input settings, their performance is fundamentally constrained by relying on a single forward pass for inference. We propose ReSplat, a feed-forward recurrent Gaussian splatting model that iteratively refines 3D Gaussians without explicitly computing gradients. Our key insight is that the Gaussian splatting rendering error serves as a rich feedback signal, guiding the recurrent network to learn effective Gaussian updates. This feedback signal naturally adapts to unseen data distributions at test time, enabling robust generalization across datasets, view counts and image resolutions.
To initialize the recurrent process, we introduce a compact reconstruction model that operates in a $16 \times$ subsampled space, producing $16 \times$ fewer Gaussians than previous per-pixel Gaussian models.
This substantially reduces computational overhead and allows for efficient Gaussian updates.
Extensive experiments across varying of input views (2, 8, 16, 32), resolutions ($256 \times 256$ to $540 \times 960$), and datasets (DL3DV, RealEstate10K and ACID) demonstrate that our method achieves state-of-the-art performance while significantly reducing the number of Gaussians and improving the rendering speed.
Our code and models will be public.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 7229