Lifelong Generative Modeling

Jason Ramapuram, Magda Gregorova, Alexandros Kalousis

Abstract: Lifelong learning is the problem of learning multiple consecutive tasks in a sequential manner where knowledge gained from previous tasks is retained and used for future learning. It is essential towards the development of intelligent machines that can adapt to their surroundings. In this work we focus on a lifelong learning approach to generative modeling where we continuously incorporate newly observed streaming distributions into our learnt model. We do so through a student-teacher architecture which allows us to learn and preserve all the distributions seen so far without the need to retain the past data nor the past models. Through the introduction of a novel cross-model regularizer, the student model leverages the information learnt by the teacher, which acts as a summary of everything seen till now. The regularizer has the additional benefit of reducing the effect of catastrophic interference that appears when we learn over streaming data. We demonstrate its efficacy on streaming distributions as well as its ability to learn a common latent representation across a complex transfer learning scenario.

TL;DR: Lifelong distributional learning through a student-teacher architecture coupled with a cross model posterior regularizer.

Keywords: Lifelong, Generative Modeling, Variational Autoencoder, VAE, Catastrophic Interference

Code: [ jramapuram/LifelongVAE_pytorch](https://github.com/jramapuram/LifelongVAE_pytorch)

Data: [Fashion-MNIST](https://paperswithcode.com/dataset/fashion-mnist), [SVHN](https://paperswithcode.com/dataset/svhn)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/lifelong-generative-modeling/code)