Class Incremental Continual Learning with Self-Organizing Maps and Synthetic Replay

ICLR 2026 Conference Submission20029 Authors

Keywords: Continual Learning, Unsupervised Continual Learning, Class Incremental Learning, Lifelong Learning, Self Organizing Maps, Autoencoder, Generative Replay

TL;DR: Unsupervised class incremental learning and synthetic replay using self-organizing maps.

Abstract: This work introduces a novel generative continual learning framework based on self-organizing maps (SOMs) extended with learned distributional statistics and encoder--decoder models which enable memory-efficient replay, eliminating the need to store raw data samples or task labels. For high-dimensional input spaces, the SOM operates over the latent space of the encoder--decoder, whereas, for lower-dimensional inputs, the SOM operates in a standalone fashion. Our method stores a running mean, variance, and covariance for each SOM unit, from which synthetic samples are then generated during future learning iterations. For the encoder--decoder method, generated samples are then fed through the decoder to then be used in subsequent replay. Experimental results on standard class-incremental benchmarks show that our approach performs competitively with state-of-the-art memory-based methods and outperforms memory-free methods, notably improving over the best state-of-the-art single class incremental performance without pretrained encoders on CIFAR-10 and CIFAR-100 by nearly $10$\% and $7$\%, respectively. We also find best performance on single class incremental CIFAR-100 utilizing a foundational encoder--decoder, and present the first baseline results for single class incremental TinyImageNet. Our methodology facilitates easy visualization of the learning process and can also be utilized as a generative model post-training. Results show our method's capability as a scalable, task-label-free, and memory-efficient solution for continual learning.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 20029