MARS: Meta-learning as score matching in the function space

Krunoslav Lehman Pavasovic, Jonas Rothfuss, Andreas Krause

Keywords: Meta-Learning, Function Space, Transfer, Score Estimation

TL;DR: Meta-learning in the function space by estimating the score function of the data-generating process marginals.

Abstract: We approach meta-learning through the lens of functional Bayesian neural network inference which views the prior as a stochastic process and performs inference in the function space. Specifically, we view the meta-training tasks as samples from the data-generating process and formalize meta-learning as empirically estimating the law of this stochastic process. Our approach can seamlessly acquire and represent complex prior knowledge by meta-learning the score function of the data-generating process marginals. In a comprehensive benchmark, we demonstrate that our method achieves state-of-the-art performance in terms of predictive accuracy and substantial improvements in the quality of uncertainty estimates.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2210.13319/code)