Information-theoretic Online Memory Selection for Continual Learning

Shengyang Sun, Daniele Calandriello, Huiyi Hu, Ang Li, Michalis Titsias

Keywords: Task-free continual learning, replay memory, information theoretic, reservoir sampling

Abstract: A challenging problem in task-free continual learning is the online selection of a representative replay memory from data streams. In this work, we investigate the online memory selection problem from an information-theoretic perspective. To gather the most information, we propose the \textit{surprise} and the \textit{learnability} criteria to pick informative points and to avoid outliers. We present a Bayesian model to compute the criteria efficiently by exploiting rank-one matrix structures. We demonstrate that these criteria encourage selecting informative points in a greedy algorithm for online memory selection. Furthermore, by identifying the importance of \textit{the timing to update the memory}, we introduce a stochastic information-theoretic reservoir sampler (InfoRS), which conducts sampling among selective points with high information. Compared to reservoir sampling, InfoRS demonstrates improved robustness against data imbalance. Finally, empirical performances over continual learning benchmarks manifest its efficiency and efficacy.

One-sentence Summary: We present information-theoretic algorithms to tackle the online memory selection problem in task-free and data imbalanced continual learning.