Understanding Local Linearisation in Variational Gaussian Process State Space Models

Talay M Cheema

Abstract:

We describe variational inference approaches in
Gaussian process state space models in terms of
local linearisations of the approximate posterior
function. Most previous approaches have either
assumed independence between the posterior dynamics and latent states (the mean-field (MF) approximation), or optimised free parameters for
both, leading to limited scalability. We use our
framework to prove that (i) there is a theoretical
imperative to use non-MF approaches, to avoid
excessive bias in the process noise hyperparameter estimate, and (ii) we can parameterise only
the posterior dynamics without any less of performance. Our approach suggests further approximations, based on the existing rich literature on filtering and smoothing for nonlinear systems, and
unifies approaches for discrete and continuous
time models.