Meta-learning inductive biases of learning systems with Gaussian processes

Michael Y. Li, Erin Grant, Thomas L. Griffiths

TL;DR: We meta-learn explicit and implicit inductive biases of neural networks using Gaussian processes.

Abstract: Many advances in machine learning can be attributed to designing systems with inductive biases well-suited for particular tasks. However, it can be challenging to ascertain the inductive biases of a learning system, much less control them in the design process. We propose a framework to capture the inductive biases in a learning system by meta-learning Gaussian process kernel hyperparameters from its predictions. We illustrate the potential of this framework across several case studies, including investigating the inductive biases of both untrained and trained neural networks, and assessing whether a given neural network family is well-suited for a task family.

Contribution Process Agreement: Yes

Poster Session Selection: Poster session #1 (12:00 UTC+1), Poster session #2 (16:50 UTC+1), Poster session #3 (19:20 UTC+1)