GenCL: Generating Diverse Examples for Name Only Continual Learning

TMLR Paper5119 Authors

Abstract: Continual learning (CL) methods often rely on supervised data. However, in CL scenarios, where new data arrive continuously, real-time manual annotation is impractical due to high costs and training delays that hinder real-time adaptation. To alleviate this, ‘name-only’ CL setup has been proposed, requiring only the name of new concepts (e.g., classes), not the supervised samples. A recent approach tackles this setup by supplementing data with web-scraped images, but such data often suffers from issues of data imbalance, noise, and copyright. To overcome the limitations of both human supervision and webly supervision, we propose Generative name only Continual Learning (GenCL) using generative models for name-only continual learning. But naïve application of generative models results in limited diversity of generated data. Here, we enhance (i) intra-diversity, the diversity of images generated by a single model, by proposing a diverse prompt generation method that generates diverse text prompts for text-to-image models, and (ii) inter-diversity, the diversity of images generated by multiple generative models, by introducing an ensemble strategy that selects minimally overlapping samples. We empirically validate that the proposed GenCL outperforms prior arts, even a model trained with fully supervised data by large margins, in various tasks, including image recognition and multi-modal visual reasoning.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Piyush_Rai1

Submission Number: 5119