Convolutional Decoupled cVAE-GANs for Pseudo-Replay Based Continual Learning

Lucas A. M. De Alcantara, José I. S. Da Silva, Miguel L. P. C. Silva, Saulo C. S. Machado, Tsang Ing Ren

Abstract: Continual Learning is the concept of having a model able to sequentially learn to solve new tasks without losing the ability to solve previous tasks. Achieving this is challenging because neural networks usually suffer from catastrophic forgetting of the preceding tasks when they are learning new ones. To handle this issue, pseudo-replay approaches leverages the performance of generative networks using them to generate samples related to past data to serve as input to the model when it is learning new tasks. In this work, we propose an improved architecture and training strategy based on the state-of-the-art pseudo-replay IRCL method. We use a cVAE-GAN as the generative model and train it decoupled from the other components of the architecture. Also, we make use of convolutional layers for the architecture components instead of linear ones. Our experimental results show that the proposed method outperforms the state-of-the-art IRCL method by up to 10% in Average Accuracy and up to 8.3% in Average Backward Transfer on both Split MNIST and Split FashionMNIST datasets.