Activation-Based Sampling for Pixel- to Image-Level Aggregation in Weakly-Supervised Segmentation

Arvi Jonnarth, Michael Felsberg, Yushan Zhang

Abstract: Classification networks have been used in weakly-supervised semantic segmentation (WSSS) to segment objects by means of class activation maps (CAMs). However, without pixel-level annotations, they are known to (1) mainly focus on discriminative regions, and (2) to produce diffuse CAMs without well-defined prediction contours. In this work, we alleviate both problems by improving CAM learning. First, we incorporate importance sampling based on the class-wise probability mass function induced by the CAMs to produce stochastic image-level class predictions. This results in segmentations that cover a larger extent of the objects, as shown in our empirical studies. Second, we formulate a feature similarity loss term, which further improves the alignment of predicted contours with edges in the image. Furthermore, we shed new light onto the problem of WSSS by measuring the contour F-score as a complement to the common area mIoU metric. We show that our method significantly outperforms previous methods in terms of contour quality, while matching state-of-the-art on region similarity.