PartCom: Part Composition Learning for 3D Open-Set Recognition

Tingyu Weng, Jun Xiao, Hao Pan, Haiyong Jiang

Abstract: In this work, we address 3D open-set recognition (OSR) that can recognize known classes as well as be aware of unknown classes during testing. The key challenge of 3D OSR is that unknown objects are not available during training and 3D closed set recognition methods trained on known classes usually classify an unknown object as a known one with high confidence. This over-confidence is mainly due to the fact that local part information in 3D shapes provides the main evidence for known class recognition, which nevertheless leads to the incorrect recognition of unknown classes that have similar local parts but arranged very differently. To address this problem, we propose PartCom, a 3D OSR method that calls attention to not only part information but also the part composition that is unique to each class. PartCom uses a part codebook to learn the different parts across object classes, and represents part composition as a latent distribution over the codebook. In this way, both known classes and unknown classes are cast into the space of learned parts, but known classes have composites largely distinguished from unknown ones, which enables OSR. To learn the part codebook, we formulate two necessary constraints to ensure the part codebook encodes diverse parts of different classes compactly and efficiently. In addition, we propose an optional augmenting module of Part-aware Unknown feaTure Synthesis, that further reduces open-set misclassification risks by synthesizing novel part compositions to be regarded as unknown classes. This synthesis is simply achieved by mixing part codes of different classes; training with such augmented data makes classifiers’ decision boundaries more closely fit the known classes and therefore improves open-set recognition. To evaluate the proposed method, we construct four 3D OSR tasks based on datasets of CAD shapes, multi-view scanned shapes, and LiDAR scanned shapes. Extensive experiments show that our method achieves significantly superior results than SOTA baselines on all tasks.