3DCOMPAT++: An improved Large-scale 3D Vision Dataset for Compositional Recognition

Habib Slim

Abstract:

In this work, we present 3DCOMPAT++ , a multi-
modal 2D/3D dataset with 160 million rendered views of more
than 10 million stylized 3D shapes carefully annotated at the part-
instance level, alongside matching RGB point clouds, 3D textured
meshes, depth maps, and segmentation masks. 3DCOMPAT++
covers 42 shape categories, 275 fine-grained part categories, and
293 fine-grained material classes that can be compositionally
applied to parts of 3D objects. We render a subset of one million
stylized shapes from four equally spaced views as well as four
randomized views, leading to a total of 160 million renderings.
Parts are segmented at the instance level, with coarse-grained
and fine-grained semantic levels. We introduce a new task,
called Grounded CoMPaT Recognition (GCR), to collectively
recognize and ground compositions of materials on parts of
3D objects. Additionally, we report the outcomes of a data
challenge organized at the CVPR conference, showcasing the
winning method’s utilization of a modified PointNet++ model
trained on 6D inputs, and exploring alternative techniques for
GCR enhancement. We hope our work will help ease future
research on compositional 3D Vision. The dataset and code have
been made publicly available at https://3dcompat-dataset.org/v2/.