Dual Parameterization of Sparse Variational Gaussian Processes

Vincent ADAM, Paul Edmund Chang, Mohammad Emtiyaz Khan, Arno Solin

Keywords: Gaussian processes, sparse variational inference, natural gradients

TL;DR: Leveraging dual-parameterization for efficient inference and learning of hyperparameters in sparse variational GP models

Abstract: Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by using a dual parameterization where each data example is assigned dual parameters, similarly to site parameters used in expectation propagation. Our dual parameterization speeds-up inference using natural gradient descent, and provides a tighter evidence lower bound for hyperparameter learning. The approach has the same memory cost as the current SVGP methods, but it is faster and more accurate.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://github.com/AaltoML/t-SVGP

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2111.03412/code)