Discrete InfoMax Codes for Meta-Learning

Yoonho Lee, Wonjae Kim, Seungjin Choi

Keywords: meta-learning, generalization, discrete representations

Abstract: This paper analyzes how generalization works in meta-learning. Our core contribution is an information-theoretic generalization bound for meta-learning, which identifies the expressivity of the task-specific learner as the key factor that makes generalization to new datasets difficult. Taking inspiration from our bound, we present Discrete InfoMax Codes (DIMCO), a novel meta-learning model that trains a stochastic encoder to output discrete codes. Experiments show that DIMCO requires less memory and less time for similar performance to previous metric learning methods and that our method generalizes particularly well in a challenging small-data setting.

TL;DR: We derive a generalization bound for meta-learning, and propose a meta-learning model that generalizes well

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1905.11656/code)