Variational Continual Bayesian Meta-LearningDownload PDF

May 21, 2021 (edited Jan 22, 2022)NeurIPS 2021 PosterReaders: Everyone
  • Keywords: meta-learning, non-stationary task distribution, dynamic Gaussian mixture model, structural variational inference
  • TL;DR: A Variational Continual Bayesian Meta-Learning algorithm aims to deal with non-stationary task distributions in an online setting.
  • Abstract: Conventional meta-learning considers a set of tasks from a stationary distribution. In contrast, this paper focuses on a more complex online setting, where tasks arrive sequentially and follow a non-stationary distribution. Accordingly, we propose a Variational Continual Bayesian Meta-Learning (VC-BML) algorithm. VC-BML maintains a Dynamic Gaussian Mixture Model for meta-parameters, with the number of component distributions determined by a Chinese Restaurant Process. Dynamic mixtures at the meta-parameter level increase the capability to adapt to diverse tasks due to a larger parameter space, alleviating the negative knowledge transfer problem. To infer posteriors of model parameters, compared to the previously used point estimation method, we develop a more robust posterior approximation method -- structured variational inference for the sake of avoiding forgetting knowledge. Experiments on tasks from non-stationary distributions show that VC-BML is superior in transferring knowledge among diverse tasks and alleviating catastrophic forgetting in an online setting.
  • Supplementary Material: pdf
  • Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
  • Code: https://github.com/QiangAIResearcher/VC-BML/tree/main
12 Replies

Loading