Representation Learning via Consistent Assignment of Views over Random Partitions

Thalles Santos Silva, Adín Ramírez Rivera

Keywords: representation learning, unsupervised learning, self-supervised learning, computer vision

Abstract:

We introduce Consistent Assignment of views over Random Partitions (CARP), a self-supervised clustering method for representation learning of visual features.
CARP learns prototypes in an end-to-end online fashion using gradient descent without additional non-differentiable modules to solve the cluster assignment problem.
We present a new pretext task based on random partitions of prototypes by enforcing consistency between views' assignments over these random subsets.
We use a fast (student) and a slow (teacher) learners to provide stable targets for the assignment task.
We present an extensive ablation study and show that our proposed random partition pretext task (1) improves the quality of the learned representations by devising multiple random classification tasks and (2) improves training stability and prevents collapsed solutions in joint-embedding training.
CARP achieves top-1 linear accuracy of 71.7% and k-NN performance of 64.8% on the ImageNet-1M, surpassing contemporary work under limited training conditions.
When trained for longer epochs, CARP outperforms state-of-the-art methods in the k-NN evaluation and performs comparably in other benchmarks.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning

TL;DR: An unsupervised representation learning method for visual data based on self-supervised clustering.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2310.12692/code)