Joint Contrastive Learning for Image Clustering

Yuxuan Yan, Na Lu, Ruofan Yan

Abstract: Recent advances in contrastive learning promote the research of many downstream tasks, especially deep clustering, which explores the potential semantic connections for unlabeled samples. However, these contrastive-based clustering methods focus on positive pairs in the pairwise contrastive framework and ignore the latent semantic relations of negative pairs, causing semantic information distortion in embedding space. In this paper, we propose joint distribution contrastive learning (JCL), an unsupervised image clustering method encoding semantic structures of negative pairs into the learned embedding space. Specifically, JCL introduces latent class variables and model discrimination task as a maximum average class conditional likelihood estimation to encourage negative pairs with the same semantic information to be closer in embedding space. The proposed joint contrastive loss of JCL is the negative-wise contrastive loss and serves as the objective function of deep clustering. JCL is a simple end-to-end online deep contrastive clustering method that jointly exploits the positive and negative pairs and synchronously learns representation and clustering to optimize the network. Extensive experiments on moderate-scale image clustering benchmarks demonstrate JCL remarkably outperforms the state-of-the-art methods.