Efficient Approaches to Gaussian Process Classification

Lehel Csató, Ernest Fokoué, Manfred Opper, Bernhard Schottky, Ole Winther

Abstract: We present three simple approximations for the calculation of the posterior mean in Gaussian Process classification. The first two methods are related to mean field ideas known in Statistical Physics. The third approach is based on Bayesian online approach which was motivated by recent results in the Statistical Mechanics of Neural Networks. We present simulation results showing: 1. that the mean field Bayesian evidence may be used for hyperparameter tuning and 2. that the online approach may achieve a low training error fast.