Diffusion Models in Space and Time via the Discretized Heat Equation

Anonymous

Keywords: diffusion, sde, generative model, pde

TL;DR: Inspired by the heat equation, we propose a new noising process for diffusion models which evolves jointly in space and time.

Abstract: We propose a new class of diffusion models which use noising processes that diffuse jointly in space and time. These noising processes evolve according to a stochastic differential equation (SDE) inspired by the heat equation, a canonical space-time diffusion. We show that sampling from the diffusion’s transition density and evaluating its score remain tractable in the Fourier domain. This approach smooths the sequence of distributions that bridge noise and data, decaying high-frequency information before the lower frequencies that encode the large-scale structure of the image. We evaluate these models on MNIST and find that they generate convincing samples.