Contrastive Gaussian Clustering for Weakly Supervised 3D Scene Segmentation

Myrna C. Silva, Mahtab Dahaghin, Matteo Toso, Alessio Del Bue

Abstract: 3D scene segmentation is a crucial task in Computer Vision, with applications in autonomous driving, augmented reality, and robotics. Traditional methods often struggle to provide consistent and accurate segmentation across different viewpoints. To address this, we look at the growing field of novel view synthesis. Methods like NeRF and 3DGS take a set of images and implicitly learn a multi-view consistent representation of the geometry of the scene; the same strategy can be extended to learn a 3D segmentation of the scene that is consistent with the 2D segmentation of an initial training set of input images.