Multimedia Generative Modelling with High-Order Langevin Dynamics

Ziqiang Shi, Rujie Liu

Abstract: Diffusion generative models based on stochastic differential equations (SDEs) with score matching have shown remarkable success in data generation. This paper introduces an advanced generative modeling approach, leveraging high-order Langevin dynamics (HOLD) coupled with score matching. Our method substantiated by third-order Langevin dynamics, extends traditional SDEs like variance exploding or variance preserving SDEs for single-variable (data) processes. HOLD uniquely models position, velocity, and acceleration, enhancing both the quality and speed of data generation. Comprising an Ornstein-Uhlenbeck process and two Hamiltonians, HOLD significantly reduces mixing time by approximately two orders of magnitude. Empirical tests on unconditional image generation using the public CIFAR-10 and ImageNet datasets demonstrate notable improvements. The proposed method achieves state-of-the-art Frechet Inception Distances of 1.85 and 1.48 on CIFAR-10 and ImageNet respectively, and also showing substantial gains in negative log-likelihood.