Unsupervised Salient Object Detection with Spectral Cluster Voting

Gyungin Shin, Samuel Albanie, Weidi Xie

Abstract: In this paper, we tackle the challenging task of unsupervised salient object detection (SOD) by leveraging spectral clustering on self-supervised features. We make the following contributions: (i) We revisit spectral clustering and demonstrate its potential to group the pixels of salient objects across various self-supervised features, e.g., Mo-Cov2, SwAV, and DINO; (ii) Given mask proposals from multiple applications of spectral clustering on image features computed from different self-supervised models, we propose a simple but effective winner-takes-all voting mechanism for selecting the salient masks, leveraging object priors based on framing and distinctiveness; (iii) Using the selected object segmentation as pseudo groundtruth masks, we train a salient object detector, termed SELF-MASK, which outperforms prior approaches on three un-supervised SOD benchmarks. Code is publicly available at https://github.com/NoelShin/selfmask.