Semi-Supervised Contrastive Learning with Orthonormal Prototypes

ICLR 2026 Conference Submission15298 Authors

Keywords: Contrastive learning, Semi-Supervised learning

Abstract: Contrastive learning has emerged as a powerful method in deep learning, excelling at learning effective representations through contrasting samples from different distributions. However, dimensional collapse, where embeddings converge into a lower-dimensional space, poses a significant challenge, especially in semi-supervised and self-supervised setups. In this paper, we propose CLOP, a novel semi-supervised loss function designed to prevent dimensional collapse by promoting the formation of orthogonal linear subspaces among class embeddings. Through extensive experiments on real and synthetic datasets, we demonstrate that CLOP improves performance in image classification and object detection tasks while also exhibiting greater stability across different learning rates and batch sizes.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 15298