ObjectCarver: Semi-automatic segmentation, reconstruction and separation of 3D objects

Gemmechu Hassena, Jonathan Hyun Moon, Ryan M. Fujii, Andrew Yuen, Noah Snavely, Steve Marschner, Bharath Hariharan

Keywords: 3D reconstruction, Scene decomposition, Mask propagation, Occlusion, Dataset

TL;DR: Given multiview images and click points on one image, ObjectCarver decomposes scenes into separate objects, providing high-quality 3D surfaces while handling occlusion and close-contact objects. We also introduce a new benchmark dataset.

Abstract:

Implicit neural fields have made remarkable progress in reconstructing 3D surfaces from multiple images; however, they encounter challenges when it comes to separating individual objects within a scene. Previous approaches to this problem require ground-truth segmentation masks and introduce floating artifacts in occluded parts of the scene.

We address these challenges with ObjectCarver. ObjectCarver requires no ground-truth segmentation; all it needs is just a few user clicks in a single view. ObjectCarver also introduces a new loss function that prevents floaters and avoids inappropriate carving-out due to occlusion.

Finally, ObjectCarver uses a simple initialization technique that significantly speeds up the process while preserving geometric details. We demonstrate qualitatively and quantitatively on multiple datasets (including a new dataset and benchmark with complete ground-truth) that ObjectCarver produces more accurate reconstructions of each object while minimizing artifacts

Submission Number: 244