Part Segmentation of Unseen Objects using Keypoint Guidance

Shujon Naha, Md A Reza, David J. Crandall

Abstract:

While object part segmentation is useful for many ap-
plications, typical approaches require a large amount of
labeled data to train a model for good performance. To
reduce the labeling effort, weak supervision cues such as
object keypoints have been used to generate pseudo-part
annotations which can subsequently be used to train larger
models. However, previous weakly-supervised part segmen-
tation methods require the same object classes during both
training and testing. We propose a new model to use key-
point guidance for segmenting parts of novel object classes
given that they have similar structures as seen objects —
different types of four-legged animals, for example. We
show that a non-parametric template matching approach is
more effective than pixel classification for part segmenta-
tion, especially for small or less frequent parts. To evaluate
the generalizability of our approach, we introduce two new
datasets that contain 200 quadrupeds in total with both key-
point and part segmentation annotations. We show that our
approach can outperform existing models by a large mar-
gin on the novel object part segmentation task using limited
part segmentation labels during training.