Consistency and Unified Semantic Regularization for Generalized Category Discovery

Yuwei Bian, Shidong Wang, Haofeng Zhang

Keywords: deep clustering; category discovery; representation learning

Abstract: Generalized Category Discovery (GCD) aims to leverage labeled data to learn clustering-friendly representations for unlabeled data. Among existing approaches, self-supervised contrastive learning (CL) is the most widely adopted, typically optimizing two objectives: $\texttt{consistency}$ and $\texttt{uniformity}$. However, we observe an inherent tension between these objectives—while uniformity encourages a uniform distribution across the feature space, it can conflict with the goal of learning class-discriminative representations. To address this, we propose a two-stage framework that disentangles feature learning from self-contrastive objectives to better capture category concepts and represent auxiliary unlabeled data. In the first stage, the model constructs visual representations anchored to known category prototypes while reinforcing semantic links between labeled classes. The second stage extends this representation space to discover novel categories using a consistency objective combined with specifically designed regularization. Moreover, we introduce a novel $\texttt{Semantic Exploration Energy mechanism}$ to capture shared semantics across categories, thereby mitigating the information loss caused by prototype orthogonalization. The proposed framework—Consistency and Unified Semantic Regularization ($\texttt{CURE}$)—retains the consistency objective and enhances it with semantic energy regularization. Our CURE achieves state-of-the-art performance across multiple benchmarks and significantly alleviates performance imbalance between known and novel classes.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 7105