Approximate natural gradient in Gaussian processes with non-log-concave likelihoods

Marcelo Hartmann

Track: Extended abstract

Keywords: Gaussian processes, natural gradient, maximum a posterior estimate, geometry, student-t model

TL;DR: Important pratical aspects in understanding the choice of Fisher information (metric-tensor) for natural gradients in gaussian processes and more generally.

Abstract: Approximate Bayesian inference on Gaussian process models with non-log-concave likelihoods is a challenging problem. When the log-likelihood function lacks concavity, finding the maximum a posterior estimate of the Gaussian process posterior becomes troublesome. Additionally, the lack of concavity complicates computer implementations and may increase computational load. In this work, we propose using an approximate Fisher information matrix as an alternative for defining a variant of the natural gradient update in the context of Gaussian process modeling, achieving this without incurring additional costs and with less analytical derivations. Moreover, experiments show that the approximate natural gradient works efficiently when the log-likelihood function strongly lacks concavity.

Submission Number: 87