Gradient Imbalance and solution in Online Continual learning

Yiduo Guo, Dongyan Zhao, Bing Liu

Keywords: Online continual learning, Lifelong learning, Gradient imbalance

Abstract: Most existing techniques for online continual learning are based on experience-replay. In this approach, a memory buffer is used to save some data from past tasks for dealing with catastrophic forgetting. In training, a small batch of data from the data stream of the current task and some sampled data from a memory buffer are used jointly to update or train the current model. In this paper, we study the experience replay-based approach from a new angle, gradient imbalance. We first investigate and analyze this phenomenon experimentally from two perspectives: imbalance of samples introduced by experience replay and sequence of classes introduced by incremental learning. To our knowledge, this problem has not been studied before and it significantly limits the performance of online continual learning. Based on observations from experiments and theoretical analysis, a new learning strategy and a new loss are proposed to deal with the problem. Empirical evaluation shows that GAD helps improve the online CL performance by more than 11% in accuracy.

One-sentence Summary: This paper analyzes the gradient imbalance problem in online continual learning and proposes a new strategy and a novel loss function to deal with the problem.