Diffusion at absolute zero: Langevin sampling using successive moreau envelopes

Andreas Habring, Martin Zach, Thomas Pock, Alexander Falk

Abstract: We propose a method for sampling from Gibbs distributions of the form $\pi(x)\propto\exp(-U(x))$ that leverages a family $(\pi^{t})_t$ of approximations of the target density which is deliberately constructed such that $\pi^{t}$ exhibits favorable properties for sampling when $t$ is large, and such that $\pi^{t}$ approaches $\pi$ as $t$ approaches 0. This sequence is obtained by replacing (parts of) the potential $U$ with its Moreau envelope. Through the sequential sampling from $\pi^{t}$ for decreasing values of $t$ by a Langevin algorithm with appropriate step size, the samples are guided from a simple starting density to the more complex target quickly. We prove that $ t \mapsto \pi^t $ is Lipschitz continuous in the total variation distance and Hölder continuous in the Wasserstein-$p$ distance, that the sampling algorithm is ergodic, and that it converges to the target density without assuming convexity or differentiability of the potential $U$. In addition to the theoretical analysis, we show experimental results that support the superiority of the method in terms of convergence speed and mode-coverage of multi-modal densities over current algorithms. The experiments range from one-dimensional toy-problems to high-dimensional inverse imaging problems with learned potentials.