Score-Based Generative Models with Lévy Processes

Eunbi Yoon, Keehun Park, Jinhyeok Kim, Sungbin Lim

Keywords: Lévy process, Stochastic Differential Equation, Markov Process

TL;DR: We derive a time reversal formula for the SDEs with Lévy processes and propose a new score-based generative models with a broad range of non-Gaussian Markov processes.

Abstract:

Time reversibility of stochastic processes is a primary cornerstone of the score-based generative models through stochastic differential equations (SDEs).
While a broader class of Markov processes is reversible, previous continuous-time approaches restrict the range of noise processes to Brownian motion (BM) since the closed-form of the time reversal formula is only known for diffusion processes.
In this paper, to expand the class of noise distribution, we propose a class of score-based probabilistic generative models, Lévy-Itō Model (LIM), which utilizes $\alpha$-stable distribution for noise injection.
To this end, we derive an approximate time reversal formula for the SDEs with Lévy processes that can allow discontinuous pure jump motion.
Consequently, we advance the score-based generative models with a broad range of non-Gaussian Markov processes.
Empirical results on MNIST, CIFAR-10, CelebA, and CelebA-HQ show that our approach is valid.

Student Paper: Yes