Semi-supervised semantic segmentation needs strong, high-dimensional perturbations

Geoff French, Timo Aila, Samuli Laine, Michal Mackiewicz, Graham Finlayson

TL;DR: Why semi-supervised semantic segmentation is a challenging problem (no cluster assumption) and how to get consistency regularisation to work

Abstract:

Consistency regularization describes a class of approaches that have yielded ground breaking results in semi-supervised classification problems. Prior work has established the cluster assumption\,---\,under which the data distribution consists of uniform class clusters of samples separated by low density regions\,---\,as key to its success. We analyze the problem of semantic segmentation and find that the data distribution does not exhibit low density regions separating classes and offer this as an explanation for why semi-supervised segmentation is a challenging problem.
We then identify the conditions that allow consistency regularization to work even without such low-density regions.
This allows us to generalize the recently proposed CutMix augmentation technique to a powerful masked variant, CowMix,
leading to a successful application of consistency regularization in the semi-supervised semantic segmentation setting and
reaching state-of-the-art results in several standard datasets.

Keywords: computer vision, semantic segmentation, semi-supervised, consistency regularisation