A Simple, Fast Algorithm for Continual Learning from High-Dimensional Data

Neil Ashtekar, Vasant G Honavar

Keywords: Continual Learning, Lifelong Learning, Incremental Learning, Adaptive Resonance Theory, Dimensionality Reduction

TL;DR: We propose a simple, fast algorithm for continual learning based on adaptive resonance theory and incremental PCA.

Abstract: As an alternative to resource-intensive deep learning approaches to the continual learning problem, we propose a simple, fast algorithm inspired by adaptive resonance theory (ART). To cope with the curse of dimensionality and avoid catastrophic forgetting, we apply incremental principal component analysis (IPCA) to the model's previously learned weights. Experiments show that this approach approximates the performance achieved using static PCA and is competitive with continual deep learning methods. Our implementation is available on https://github.com/neil-ash/ART-IPCA