PseudoSeg: Designing Pseudo Labels for Semantic Segmentation

Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, Tomas Pfister

Keywords: pseudo-labeling, semi-supervised, semantic-segmentation

Abstract: Recent advances in semi-supervised learning (SSL) demonstrate that a combination of consistency regularization and pseudo-labeling can effectively improve image classification accuracy in the low-data regime. Compared to classification, semantic segmentation tasks require much more intensive labeling costs. Thus, these tasks greatly benefit from data-efficient training methods. However, structured outputs in segmentation render particular difficulties (e.g., designing pseudo-labeling and augmentation) to apply existing SSL strategies. To address this problem, we present a simple and novel re-design of pseudo-labeling to generate well-calibrated structured pseudo labels for training with unlabeled or weakly-labeled data. Our proposed pseudo-labeling strategy is network structure agnostic to apply in a one-stage consistency training framework. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes. Extensive experiments have validated that pseudo labels generated from wisely fusing diverse sources and strong data augmentation are crucial to consistency training for segmentation. The source code will be released.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: This paper presents a new method that first demonstrates how well-calibrated soft pseudo labels obtained through wise fusion of predictions from diverse sources greatly improve consistency training for semantic segmentation.

Code: [ googleinterns/wss](https://github.com/googleinterns/wss) + [ 1 community implementation](https://paperswithcode.com/paper/?openreview=-TwO99rbVRu)

Data: [MS COCO](https://paperswithcode.com/dataset/coco), [VOC 2012](https://paperswithcode.com/dataset/voc-2012)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2010.09713/code)