Weighting Pseudo-labels via High-Activation Feature Index Similarity and Object Detection for Semi-supervised Segmentation

Prantik Howlader, Hieu Le, Dimitris Samaras

Abstract:

Semi-supervised semantic segmentation methods leverage un-
labeled data by pseudo-labeling them. Thus the success of these methods
hinges on the reliablility of the pseudo-labels. Existing methods mostly
choose high-confidence pixels in an effort to avoid erroneous pseudo-
labels.However,highconfidencedoesnotguaranteecorrectpseudo-labels
especially in the initial training iterations. In this paper, we propose a
novel approach to reliably learn from pseudo-labels. First, we unify the
predictions from a trained object detector and a semantic segmentation
model to identify reliable pseudo-label pixels. Second, we assign different
learning weights to pseudo-labeled pixels to avoid noisy training signals.
To determine these weights, we first use the reliable pseudo-label pixels
identified from the first step and labeled pixels to construct a prototype
for each class. Then, the per-pixel weight is the structural similarity
between the pixel and the prototype measured via rank-statistics simi-
larity. This metric is robust to noise, making it better suited for compar-
ing features from unlabeled images, particularly in the initial training
phases where wrong pseudo labels are prone to occur. We show that
our method can be easily integrated into four semi-supervised semantic
segmentation frameworks, and improves them in both Cityscapes and
Pascal VOC datasets.