Gaussian Processes for Regression

Christopher K. I. Williams, Carl Edward Rasmussen

Abstract: The Bayesian analysis of neural networks is difficult because a sim(cid:173) ple prior over weights implies a complex prior distribution over functions . In this paper we investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian anal(cid:173) ysis for fixed values of hyperparameters to be carried out exactly using matrix operations. Two methods, using optimization and av(cid:173) eraging (via Hybrid Monte Carlo) over hyperparameters have been tested on a number of challenging problems and have produced excellent results.