Learnable Pixel Clustering Via Structure and Semantic Dual Constraints for Unsupervised Image Segmentation

Bo Wang, Shiang Wang, Chunfeng Yuan, Zhonghai Wu, Bing Li, Weiming Hu, Jeffrey Xiong

Abstract: Unsupervised image segmentation is a challenge task, since a high-quality segmented image should perceive not only local object structures but also certain semantics without any annotations. In this paper, we propose a novel encoder-decoder pixel clustering framework with dual constraints to incorporate local structure and global semantic information for guiding pixel feature learning in a self-supervised manner. On one hand, a Local Structure Constraint (LStC) is constructed based on fine-grained superpixels, which improves the boundary perception of pixel features by keeping intra-superpixel feature consistency and largening inter-superpixel feature distance. On the other hand, a new Global Semantic Constraint (GSeC) is proposed via adapting the mutual information maximization technique to the single-image setting, and it strengthens the global semantic perception of pixel features and thus improves the segmenting integrity of objects. Finally, based on the learned pixel features, a smoothing component is employed to achieve semantically meaningful pixel clustering. The experimental evaluation on BSDS500 and PASCAL Context datasets show the superiority of our method on region and boundary qualities.