Energetic Variational Gaussian Process Regression

Lulu Kang, Yuanxing Cheng, Yiwei Wang, Chun Liu

Abstract: The Gaussian process (GP) regression model is a widely employed supervised learning approach. In this paper, we estimate the GP model through variational inference, particularly employing the recently introduced energetic variational inference method. Under the GP model assumptions, we derive posterior distributions for its parameters. The energetic variational inference approach bridges the Bayesian sampling and optimization and enables approximation of the posterior distributions and identification of the posterior mode. By incorporating a Gaussian prior on the mean component of the GP model, we also apply shrinkage estimation to the parameters, facilitating variable selection of the mean function. The proposed GP method outperforms some existing software packages on three benchmark examples.