A Comparative Study of Continual Backpropagation

Jacopo Silvestrin, Francisco S. Melo, Manuel Lopes

Abstract: In continual learning, artificial agents need to be able to learn in a sequential fashion, adapting to new data and maintaining relevant information about the past. A growing body of research shows that neural networks tend to overfit the first samples they encounter, making it challenging to adapt to future tasks. Algorithms that periodically reset part of their network parameters have been shown to maintain the ability to continuously learn from new data. However, these methods are prone to catastrophic forgetting as they do not keep information about the past. In this paper, we provide a comparative study on continual adaptation and catastrophic forgetting capabilities of several variations of the Continual Backprop algorithm, investigating the impact of increasing the network capacity and choosing which parameters to reset based on their Fisher information.