Experiential Learning: Using Partition-Tree Weighting and MAML in the Continual and Online Setting
Abstract: Learning from experience means remaining adaptive and responsive to errors over time. However, gradient-based deep learning can fail dramatically in the continual, online setting. In this work we address this shortcoming by combining two meta-learning methods: the purely online Partition Tree Weighting (PTW) mixture-of-experts algorithm, and a novel variant of the Model-Agnostic Meta-Learning (MAML) initialization-learning procedure. We demonstrate our approach, \RMPTW, in a piecewise stationary classification task. In this continual, online setting, \RMPTW matches and even outperforms an augmented learner that is allowed to pre-train offline and is given explicit notification when the task changes. \RMPTW thus provides a base learner with the benefits of offline training, explicit task sampling, and boundary notification, all for a $O(\log_2(t))$ increase in computation and memory. This makes deep learning more viable for fully online, task-agnostic continual learning, which is at the heart of general intelligence.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Laurent_Charlin1
Submission Number: 1445
Loading