Adaptive Gradient-Based Meta-Learning Methods

Anonymous

Keywords: meta-learning, online learning

TL;DR: Practical adaptive algorithms for gradient-based meta-learning with provable guarantees.

Abstract: We build a theoretical framework for understanding practical meta-learning methods that enables the integration of sophisticated formalizations of task-similarity with the extensive literature on online convex optimization and sequential prediction algorithms in order to provide within-task performance guarantees. Our approach improves upon recent analyses of parameter-transfer by enabling the task-similarity to be learned adaptively and by improving transfer-risk bounds in the setting of statistical learning-to-learn. It also leads to straightforward derivations of average-case regret bounds for efficient algorithms in settings where the task-environment changes dynamically or the tasks share a certain geometric structure.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1906.02717/code)