Promising or Elusive? Unsupervised Object Segmentation from Real-world Single Images

Yafei YANG, Bo Yang

Keywords: Computer Vision, Image Segmentation, Unsupervised Learning

Abstract: In this paper, we study the problem of unsupervised object segmentation from single images. We do not introduce a new algorithm, but systematically investigate the effectiveness of existing unsupervised models on challenging real-world images. We firstly introduce four complexity factors to quantitatively measure the distributions of object- and scene-level biases in appearance and geometry for datasets with human annotations. With the aid of these factors, we empirically find that, not surprisingly, existing unsupervised models catastrophically fail to segment generic objects in real-world images, although they can easily achieve excellent performance on numerous simple synthetic datasets, due to the vast gap in objectness biases between synthetic and real images. By conducting extensive experiments on multiple groups of ablated real-world datasets, we ultimately find that the key factors underlying the colossal failure of existing unsupervised models on real-world images are the challenging distributions of object- and scene-level biases in appearance and geometry. Because of this, the inductive biases introduced in existing unsupervised models can hardly capture the diverse object distributions. Our research results suggest that future work should exploit more explicit objectness biases in the network design.

TL;DR: By training more than 200 models, we demonstrate that current unsupervised methods cannot segment generic objects from real-world single images, unless the complex objectness biases are removed.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2210.02324/code)