Marginalised Spectral Mixture Kernels with Nested Sampling

Fergus Simpson, Vidhi Lalchand, Carl Edward Rasmussen

Abstract: Gaussian Process (GPs) models are a rich distribution over functions with inductive biases controlled by a kernel function. Learning occurs through optimisation of the kernel hyperparameters using the marginal likelihood as the objective (ML-II). This work analyses the benefits of marginalising kernel hyperparameters using nested sampling (NS), a technique well-suited to sample from complex, multi-modal distributions. We benchmark against Hamiltonian Monte Carlo (HMC) on time-series regression tasks and find that a principled approach to quantifying hyperparameter uncertainty substantially improves the quality of prediction intervals.