Graph Random Features for Scalable Gaussian Processes

Matthew Zhang, Jihao Andreas Lin, Krzysztof Marcin Choromanski, Adrian Weller, Richard E. Turner, Isaac Reid

Keywords: kernels, graphs, Gaussian processes, Monte Carlo, inference

TL;DR: Approximating graph node kernels using random walks unlocks efficient GPs on graphs.

Abstract: We study the application of graph random features (GRFs) – a recently-introduced stochastic estimator of graph node kernels – to scalable Gaussian processes on discrete input spaces. We prove that (under mild assumptions) Bayesian inference with GRFs enjoys $\mathcal{O}(N^{3/2})$ time complexity with respect to the number of nodes $N$, with probabilistic accuracy guarantees. In contrast, exact kernels generally incur $\mathcal{O}(N^{3})$. Wall-clock speedups and memory savings unlock Bayesian optimisation with over 1M graph nodes on a single computer chip, whilst preserving competitive performance.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 6344