Learning to remember: Dynamic Generative Memory for Continual Learning

Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Moin Nabi,

Abstract: Continuously trainable models should be able to learn from a stream of data over an undefined period of time. This becomes even more difficult in a strictly incremental context, where data access to previously seen categories is not possible. To that end, we propose making use of a conditional generative adversarial model where the generator is used as a memory module through neural masking to emulate neural plasticity in the human brain. This memory module is further associated with a dynamic capacity expansion mechanism. Taken together, this method facilitates a resource efficient capacity adaption to accommodate new tasks, while retaining previously attained knowledge. The proposed approach outperforms state-of-the-art algorithms on publicly available datasets, overcoming catastrophic forgetting.

Keywords: Continual Learning, Catastrophic Forgetting, Dynamic Network Expansion

Data: [SVHN](https://paperswithcode.com/dataset/svhn)