Contextualizing Enhances Gradient Based Meta Learning

Evan Vogelbaum, Rumen Dangovski, Li Jing, Marin Soljacic

Abstract:

Meta learning methods have found success when applied to few shot classification
problems, in which they quickly adapt to a small number of labeled examples.
Prototypical representations, each representing a particular class, have been of
particular importance in this setting, as they provide a compact form to convey
information learned from the labeled examples. However, these prototypes are
just one method of representing this information, and they are narrow in their
scope and ability to classify unseen examples. We propose the implementation of
contextualizers, which are generalizable prototypes that adapt to given examples
and play a larger role in classification for gradient-based models. We demonstrate
how to equip meta learning methods with contextualizers and show that their use
can significantly boost performance on a range of few shot learning datasets. We
also present figures of merit demonstrating the potential benefits of contextualizers,
along with analysis of how models make use of them. Our approach is particularly
apt for low-data environments where it is difficult to update parameters without
overfitting. Our implementation and instructions to reproduce the experiments are
available at https://github.com/naveace/proto-context/.