Hierarchical Expert Networks for Meta-Learning

Heinke Hihn, Daniel A. Braun

Student First Author: Yes

Keywords: Meta-Learning, Regression, Classification, Reinforcement Learning, Information Theory

Previously Published: This work is part of a Journal article currently under review.

Abstract: The goal of meta-learning is to train a model on a variety of learning tasks, such that it can adapt to new problems within only a few iterations. Here we propose a principled information-theoretic model that optimally partitions the underlying problem space such that the resulting partitions are processed by specialized expert decision-makers. To drive this specialization we impose the same kind of information processing constraints both on the partitioning and the expert decision-makers. We argue that this specialization leads to efficient adaptation to new tasks. To demonstrate the generality of our approach we evaluate three meta-learning domains: image classification, regression, and reinforcement learning.

TL;DR: We introduce an information-theoretic hierarchical meta-learning algorithm based on specialized decision-makers.