Continual Generalized Category Discovery: Learning and Forgetting from a Bayesian Perspective

Hao Dai, Jagmohan Chauhan

TL;DR: VB-CGCD, a variational Bayes-based framework for Continual Generalized Category Discovery, achieved a +15.21% improvement outperforming state-of-the-art methods.

Abstract:

Continual Generalized Category Discovery (C-GCD) faces a critical challenge: incrementally learning new classes from unlabeled data streams while preserving knowledge of old classes.
Existing methods struggle with catastrophic forgetting, especially when unlabeled data mixes known and novel categories.
We address this by analyzing C-GCD’s forgetting dynamics through a Bayesian lens, revealing that covariance misalignment between old and new classes drives performance degradation.
Building on this insight, we propose Variational Bayes C-GCD (VB-CGCD), a novel framework that integrates variational inference with covariance-aware nearest-class-mean classification.
VB-CGCD adaptively aligns class distributions while suppressing pseudo-label noise via stochastic variational updates.
Experiments show VB-CGCD surpasses prior art by +15.21% with the overall accuracy in the final session on standard benchmarks.
We also introduce a new challenging benchmark with only 10% labeled data and extended online phases—VB-CGCD achieves a 67.86% final accuracy, significantly higher than state-of-the-art (38.55%), demonstrating its robust applicability across diverse scenarios. Code is available at: https://github.com/daihao42/VB-CGCD

Lay Summary:

Continual Generalized Category Discovery (C‑GCD) tackles a key problem in machine learning: how to teach a computer to recognize brand‑new categories without forgetting the ones it already knows, all while seeing only unlabeled data over time. Most existing approaches struggle because mixing old and new examples causes the system to “forget” earlier categories—a phenomenon called catastrophic forgetting.
We took a fresh look at this issue through a Bayesian lens, discovering that the culprit behind forgetting is when the computer’s internal idea of how old and new categories vary (their “covariances”) gets out of sync. To fix this, we created a new method called Variational Bayes C‑GCD (VB‑CGCD), which uses probabilistic tools to align old and new category patterns.
In tests on standard benchmarks, VB-CGCD improved the final accuracy by over 15 percent compared to the previous best. We also designed a tougher challenge—only 10 percent of the data came with labels, and VB‑CGCD achieved nearly 68 percent accuracy, showing it works reliably across varied real‑world scenarios.

Link To Code: https://github.com/daihao42/VB-CGCD

Primary Area: Applications->Computer Vision

Keywords: Continual Learning, Generalized Category Discovery, Variational Bayes

Submission Number: 3914