Adaptive Gradient-Based Meta-Learning Methods


May 16, 2019 Blind Submission readers: everyone
  • Keywords: meta-learning, online learning
  • TL;DR: Practical adaptive algorithms for gradient-based meta-learning with provable guarantees.
  • Abstract: We build a theoretical framework for understanding practical meta-learning methods that enables the integration of sophisticated formalizations of task-similarity with the extensive literature on online convex optimization and sequential prediction algorithms in order to provide within-task performance guarantees. Our approach improves upon recent analyses of parameter-transfer by enabling the task-similarity to be learned adaptively and by improving transfer-risk bounds in the setting of statistical learning-to-learn. It also leads to straightforward derivations of average-case regret bounds for efficient algorithms in settings where the task-environment changes dynamically or the tasks share a certain geometric structure.
0 Replies