Implications of Gaussian process kernel mismatch for out-of-distribution data

Beau Coker, Finale Doshi-Velez

Keywords: Gaussian processes, misspecification, kernels, out-of-distribution

TL;DR: GPs can perform poorly away from the data if the kernel is too smooth

Abstract:

Gaussian processes provide reliable uncertainty estimates in nonlinear modeling, but a poor choice of the kernel can lead to poor generalization. Although learning the hyperparameters of the kernel typically leads to optimal generalization on in-distribution test data, we demonstrate issues with out-of-distribution test data. We then investigate three potential solutions-- (1) learning the smoothness using a discrete cosine transform, (2) assuming fatter tails in function-space using a Student-$t$ process, and (3) learning a more flexible kernel using deep kernel learning--and find some evidence in favor of the first two.

Submission Number: 97