Sharp Calibrated Gaussian Processes

Alexandre Capone, Sandra Hirche, Geoff Pleiss

Keywords: Gaussian Processes, Frequentist Statistics, Kernel Methods, Model Selection and Structure Learning, Regression

TL;DR: We show how to obtain sharp calibrated Gaussian process models by exploiting kernel properties.

Abstract: While Gaussian processes are a mainstay for various engineering and scientific applications, the uncertainty estimates don't satisfy frequentist guarantees and can be miscalibrated in practice. State-of-the-art approaches for designing calibrated models rely on inflating the Gaussian process posterior variance, which yields confidence intervals that are potentially too coarse. To remedy this, we present a calibration approach that generates predictive quantiles using a computation inspired by the vanilla Gaussian process posterior variance but using a different set of hyperparameters chosen to satisfy an empirical calibration constraint. This results in a calibration approach that is considerably more flexible than existing approaches, which we optimize to yield tight predictive quantiles. Our approach is shown to yield a calibrated model under reasonable assumptions. Furthermore, it outperforms existing approaches in sharpness when employed for calibrated regression.

Submission Number: 1902