SO(2)-Equivariant Single-View 3D Reconstruction via Gaussian Sculpting Networks

Ruihan Xu, Anthony Opipari, Joshua Mah, Stanley Robert Lewis, Haoran Zhang, Hanzhe Guo, Odest Chadwicke Jenkins

Keywords: Gaussian Splatting, Single-View 3D Reconstruction, Robot Learning, Robot Manipulation

TL;DR: Fast single-view 3D reconstruction with Gaussian Splatting that enables manipulation on unseen geometry

Abstract:

This paper introduces SO(2)-Equivariant Gaussian Sculpting Networks as an approach for SO(2)-Equivariant 3D object reconstruction from single-view image observations.
Gaussian sculpting networks (GSNs) take a single observation as input to generate a Gaussian splat representation describing the observed object's geometry and texture.
By using a shared feature extractor before decoding Gaussian colors, covariances, positions, and opacities, GSNs achieve extremely high throughput (>150FPS).
Experiments demonstrate that GSNs can be trained efficiently using a multi-view rendering loss and are competitive, in quality, with expensive diffusion-based reconstruction algorithms.
Moreover, we demonstrate the potential for GSNs to be used within a robotic manipulation pipeline for object-centric grasping.

Submission Number: 1