Connectivity-constrained interactive annotations for panoptic segmentation

Ruobing Shen, Bo Tang, Ismail Ben Ayed, Andrea Lodi, Thomas Guthier

Abstract:

Large-scale ground truth data sets are of crucial importance for deep learning
based segmentation models, but annotating per-pixel
masks is prohibitively time consuming. In this paper, we investigate interactive graph-based segmentation algorithms that enforce connectivity. To be more precise, we introduce an instance-aware heuristic of a discrete Potts model, and a class-aware Integer Linear Programming (ILP) formulation that ensures global optimum. Both algorithms can take RGB, or utilize the feature maps from any DCNN, whether trained on the target dataset or not, as input. We present competitive semantic (and panoptic) segmentation results on the PASCAL VOC 2012 and Cityscapes dataset given initial scribbles. We also demonstrate that our interactive approach can reach $90.6\%$ mIoU on VOC validation set with an overhead of just $3$ correction scribbles. They are thus suitable for interactive annotation on new or existing datasets, or can be used inside any weakly supervised learning framework on new datasets.

Keywords: Panoptic Segmentation, Semantic Segmentation, Interactive Segmentation, Integer Programming