Geometric Neural Diffusion Processes

Emile Mathieu, Vincent Dutordoir, Michael John Hutchinson, Valentin De Bortoli, Yee Whye Teh, Richard E Turner

Keywords: diffusion model, functional space, stochastic process, time-series, neural processes, Gaussian processes, random fields, invariance, equivariance, symmetries, stationarity

TL;DR: We extend diffusion models to functional data with arbitrary codomain and incorporating symmetry

Abstract:

Denoising diffusion models have proven to be a flexible and effective paradigm for generative modelling.
Their recent extension to infinite dimensional Euclidean spaces has allowed for the modelling of stochastic processes.
However, many problems in the natural sciences incorporate symmetries and involve data living in non-Euclidean spaces.
In this work, we extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling.
We do so by a) constructing a noising process which admits, as limiting distribution, a geometric Gaussian process that transforms under the symmetry group of interest, and b) approximating the score with a neural network that is equivariant w.r.t. this group.
We show that with these conditions, the generative functional model admits the same symmetry.
We demonstrate scalability and capacity of the model, using a novel Langevin-based conditional sampler, to fit complex scalar and vector fields, with Euclidean and spherical codomain, on synthetic and real-world weather data.

Submission Number: 7106