Big GANs Are Watching You: Towards Unsupervised Object Segmentation with Off-the-Shelf Generative Models

Andrey Voynov, Stanislav Morozov, Artem Babenko

Keywords: GAN, segmentation, unsupervised

Abstract:

Since collecting pixel-level groundtruth data is expensive, unsupervised visual understanding problems are currently an active research topic. In particular, several recent methods based on generative models have achieved promising results for object segmentation and saliency detection. However, since generative models are known to be unstable and sensitive to hyperparameters, the training of these methods can be challenging and time-consuming.

In this work, we introduce an alternative, much simpler way to exploit generative models for unsupervised object segmentation. First, we explore the latent spaces of the publicly available unsupervised models, such as BigBiGAN and StyleGAN2, and reveal the ``segmenting'' latent directions that can be used to obtain saliency masks for GAN-produced images. These masks then are used to train a discriminative segmentation model. Being very simple and easy-to-reproduce, our approach outperforms the state-of-the-art on common benchmarks in the unsupervised scenario.
All the code and the pretrained models are available online.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: Completely unsupervised off-the-shelf GANs can be used for object segmentation without labels

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2006.04988/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=QMjJZibu86