Variational Task Encoders for Model-Agnostic Meta-Learning

Luuk Schagen, Joaquin Vanschoren

Keywords: Meta-learning, Variational Task Encoders, MAML, Multimodal

TL;DR: We extend model-agnostic meta-learning with variational inference to generalize across more divergent task distributions

Abstract: Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on novel tasks. A critical challenge lies in the inherent uncertainty about whether new tasks can be considered similar to those observed before, and robust meta-learning methods would ideally reason about this to produce corresponding uncertainty estimates. We extend model-agnostic meta-learning with variational inference: we model the identity of new tasks as a latent random variable, which modulates the fine-tuning of meta-learned neural networks. Our approach requires little additional computation and doesn't make strong assumptions about the distribution of the neural network weights, and allows the algorithm to generalize to more divergent task distributions, resulting in better-calibrated uncertainty measures while maintaining accurate predictions.

Contribution Process Agreement: Yes

Author Revision Details:

We addressed several of the clarity issues raised.
Unfortunately, we did not have the time and resources to provide the additional experiments, but hope to do so in future versions of this paper.

Poster Session Selection: Poster session #2 (16:50 UTC+1)