S$^2$GS: Self-supervised Gaussian Segmentation for Automatic 3D Object Scanning

Jinguang Tong, Xuesong Li, Sundaram Muthu, Fahira Afzal Maken, Chuong Nguyen, Hongdong Li

Keywords: Self-supervised segmentation, Gaussian splatting, 3D Reconstruction

Abstract: Automatic 3D object scanning typically involves reconstructing rotating objects from images captured from different viewpoints. In such circumstances where both the object and camera are moving, existing methods need object masks for reconstruction, and the mask quality can significantly affect the final reconstruction. However, obtaining high-quality and view-consistent object masks is challenging and laborious in practice. We address this issue by introducing Self-Supervised Gaussian Segmentation (S$^2$GS), which automatically segments the object from the background without relying on any segmentation masks. This is achieved by extending Gaussian Splatting with a learnable parameter that indicates the probability of each Gaussian belonging to the target object. We optimize this parameter using implicit object transformation constraints and regularization terms. We evaluate S$^2$GS on our new synthetic and real datasets. Experimental results show that our approach outperforms the state-of-the-art methods (2DGS) with object masks by \(27\%\) for novel-view synthesis and \(7\%\) for geometry reconstruction.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9306