Open-set Semantic Segmentation for Point Clouds via Adversarial Prototype Framework

Qiulei Dong

Abstract:

much attention in computer vision. Most of the existing
works in literature assume that the training and testing
point clouds have the same object classes, but they are generally
invalid in many real-world scenarios for identifying
the 3D objects whose classes are not seen in the training
set. To address this problem, we propose an Adversarial
Prototype Framework (APF) for handling the open-set
3D semantic segmentation task, which aims to identify 3D
unseen-class points while maintaining the segmentation
performance on seen-class points. The proposed APF
consists of a feature extraction module for extracting point
features, a prototypical constraint module, and a feature
adversarial module. The prototypical constraint module is
designed to learn prototypes for each seen class from point
features. The feature adversarial module utilizes generative
adversarial networks to estimate the distribution of unseenclass
features implicitly, and the synthetic unseen-class
features are utilized to prompt the model to learn more
effective point features and prototypes for discriminating
unseen-class samples from the seen-class ones. Experimental
results on two public datasets demonstrate that the
proposed APF outperforms the comparative methods by a
large margin in most cases.