Integration-free Kernels for Equivariant Gaussian Process Modelling

Tim Steinert, David Ginsbourger, August Lykke-Møller, Ove Christiansen, Henry Moss

TL;DR: Introducing a novel class of integration-free equivariant Gaussian process kernels, enabling efficient modelling of vector-valued random fields with applications to velocity fields and molecular dipole moment estimation with high accuracy.

Abstract:

We study the incorporation of equivariances into vector-valued GPs and more general classes of random field models.
While kernels guaranteeing equivariances have been investigated previously, their evaluation is often computationally prohibitive due to required integrations over the involved groups. In this work, we provide a kernel characterization of stochastic equivariance for centred second-order vector-valued random fields and we construct integration-free equivariant kernels based on the notion of fundamental regions of group actions. We establish data-efficient and computationally lightweight GP models for velocity fields and molecular electric dipole moments and demonstrate that proposed integration-free kernels may also be leveraged to extract equivariant components from data.

Lay Summary: Many natural and artificial systems change in predictable ways under certain transformations of their inputs. When using machine learning methods for such systems, incorporating such knowledge within the algorithms is essential to allow for efficiently leveraging available data. This is especially true when data are scarce, which calls in turn for methods providing prediction uncertainty. We focus here on Gaussian process models, which allow probabilistically modelling vector-valued outputs and have been found especially convenient in low data regimes. At the core of a Gaussian process is the kernel, which measures how similar responses are expected to be for different inputs. This allows the model to learn patterns in the data, including how the different parts of the output are related to each other. To ensure that GP models respect structural knowledge such as equivariances, the kernel itself must be suitably designed to reflect them. However, most classes of equivariant kernels rely on integration and the resulting computational burden can hinder their usability. In this work, we introduce a class of integration-free equivariant kernels which enable computationally efficient equivariant Gaussian process modelling. As shown in the paper, the proposed approach allows for efficient probabilistic predictions in challenging problems touching upon quantum chemistry and fluid dynamics, illustrating benefits of accounting for domain knowledge within machine learning.

Link To Code: https://github.com/equivariantrf/Equivariant-Random-Fields?tab=readme-ov-file

Primary Area: Probabilistic Methods->Gaussian Processes

Keywords: Gaussian processes, equivariance, kernels, molecular data

Flagged For Ethics Review: true

Submission Number: 11221