Clustering Properties of Self-Supervised Learning

Xi Weng, Jianing An, Xudong Ma, Binhang Qi, Jie Luo, Xi Yang, Jin Song Dong, Lei Huang

TL;DR: This paper proposes a novel positive-feedback SSL method, which leverages the model's clustering properties to promote learning in a self-guided manner.

Abstract: Self-supervised learning (SSL) methods via joint embedding architectures have proven remarkably effective at capturing semantically rich representations with strong clustering properties, magically in the absence of label supervision. Despite this, few of them have explored leveraging these untapped properties to improve themselves. In this paper, we provide an evidence through various metrics that the encoder's output *encoding* exhibits superior and more stable clustering properties compared to other components. Building on this insight, we propose a novel positive-feedback SSL method, termed **Re**presentation **S**elf-**A**ssignment (ReSA), which leverages the model's clustering properties to promote learning in a self-guided manner. Extensive experiments on standard SSL benchmarks reveal that models pretrained with ReSA outperform other state-of-the-art SSL methods by a significant margin. Finally, we analyze how ReSA facilitates better clustering properties, demonstrating that it effectively enhances clustering performance at both fine-grained and coarse-grained levels, shaping representations that are inherently more structured and semantically meaningful.

Lay Summary:

Computers can now teach themselves by simply looking at huge piles of data, ­without anyone telling them what each piece is. These “self-taught” systems naturally sort what they see into loose groups, a bit like stacking photos into family albums. Strangely, most existing methods ignore those home-made albums instead of using them to learn even better.

Our work asks: What if the computer paid attention to its own sorting? First, we show that the raw groups formed inside the network are surprisingly neat and stable. Then we introduce ReSA — Representation Self-Assignment. ReSA is a simple method that lets the network look at its own groups and refine them further, giving itself instant feedback, much like a student who double-checks their notes while studying.

Across standard image benchmarks, ReSA-trained models consistently beat the best previous self-learning methods. The result is a smarter system that learns faster, organizes information more clearly, and needs no extra human labels.

Link To Code: https://github.com/winci-ai/resa

Primary Area: Deep Learning->Self-Supervised Learning

Keywords: self-supervised learning, joint embedding architectures, clustering properties, positive-feedback learning

Submission Number: 240