Spatio-Temporal Variational Gaussian Processes

Oliver Hamelijnck, William J. Wilkinson, Niki Andreas Loppi, Arno Solin, Theo Damoulas

Keywords: Gaussian processes, Variational Inference, Spatio-Temporal Analysis

Abstract: We introduce a scalable approach to Gaussian process inference that combines spatio-temporal filtering with natural gradient variational inference, resulting in a non-conjugate GP method for multivariate data that scales linearly with respect to time. Our natural gradient approach enables application of parallel filtering and smoothing, further reducing the temporal span complexity to be logarithmic in the number of time steps. We derive a sparse approximation that constructs a state-space model over a reduced set of spatial inducing points, and show that for separable Markov kernels the full and sparse cases exactly recover the standard variational GP, whilst exhibiting favourable computational properties. To further improve the spatial scaling we propose a mean-field assumption of independence between spatial locations which, when coupled with sparsity and parallelisation, leads to an efficient and accurate method for large spatio-temporal problems.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

TL;DR: We derive a natural gradient variational inference method for Gaussian processes based on filtering and smoothing that improves the computational efficiency and predictive performance when applied to large spatio-temporal data.

Code: https://github.com/AaltoML/spatio-temporal-GPs

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2111.01732/code)