Hierarchical Parametrization with Gaussian Process for Bayesian Meta-Learning

Yifei Wu, Xue Zhao, Yifeng Yang, Qinying Gu, Xinbing Wang, Nanyang Ye

Keywords: Bayesian meta-learning, Bayesian Hyper-networks, Probabilistic Modeling Methods

Abstract: Meta-learning has emerged as a key approach to preparing neural networks for deployment with limited training data. Mainstream solutions focus on parameter initialization across training episodes to enable rapid generalization, which can be interpreted as adjusting the episode-specific posterior based on a cross-episode prior distribution in the context of Bayesian meta-learning. Despite the demonstrated efficacy of this probabilistic meta-learning paradigm, existing methodologies encounter performance bottlenecks, particularly as the scale and number of episodes increase. A promising strategy involves the integration of a hyper-network, which establishes a parameter memorization space across diverse episodes. In this paper, we propose Hierarchical Parametrization with Gaussian Process (HP-GP), a novel probabilistic meta-learning method that leverages the power of Gaussian Process. By implementing the amortization network layer-wise with decoupling variational Gaussian Process and normalizing flow, HP-GP offers probabilistic parametrization for meta-learning while requiring minimal modifications to the network architecture. This enables flexible and scalable integration of meta-learning into existing neural networks. Our experiments demonstrate the flexibility and robust generalization of HP-GP, outperforming other popular meta-learning methods.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 11913