Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation

Yaling Tao, Kentaro Takagi, Kouta Nakata

Keywords: clustering, representation learning, deep embedding

Abstract: Clustering is one of the most fundamental tasks in machine learning. Recently, deep clustering has become a major trend in clustering techniques. Representation learning often plays an important role in the effectiveness of deep clustering, and thus can be a principal cause of performance degradation. In this paper, we propose a clustering-friendly representation learning method using instance discrimination and feature decorrelation. Our deep-learning-based representation learning method is motivated by the properties of classical spectral clustering. Instance discrimination learns similarities among data and feature decorrelation removes redundant correlation among features. We utilize an instance discrimination method in which learning individual instance classes leads to learning similarity among instances. Through detailed experiments and examination, we show that the approach can be adapted to learning a latent space for clustering. We design novel softmax-formulated decorrelation constraints for learning. In evaluations of image clustering using CIFAR-10 and ImageNet-10, our method achieves accuracy of 81.5% and 95.4%, respectively. We also show that the softmax-formulated constraints are compatible with various neural networks.

One-sentence Summary: We present a clustering-friendly representation learning method using instance discrimination and feature decorrelation, which achieves accuracy of 81.5% and 95.4% on CIFAR-10 and ImageNet-10, respectively, far above state-of-the-art values.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Code: [ TTN-YKK/Clustering_friendly_representation_learning](https://github.com/TTN-YKK/Clustering_friendly_representation_learning)

Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10), [CIFAR-100](https://paperswithcode.com/dataset/cifar-100), [ImageNet](https://paperswithcode.com/dataset/imagenet), [STL-10](https://paperswithcode.com/dataset/stl-10)