Non-asymptotic Error Bounds in $\mathcal{W}_2$-Distance with Sqrt(d) Dimension Dependence and First Order Convergence for Langevin Monte Carlo beyond Log-Concavity

Bin Yang, Xiaojie Wang

TL;DR: We prove an optimal error bound $O(\sqrt{d} h)$ in $\mathcal{W}_2$-distance with sqrt(d) dimension dependence for Langevin Monte Carlo beyond log-Concavity.

Abstract: Generating samples from a high dimensional probability distribution is a fundamental task with wide-ranging applications in the area of scientific computing, statistics and machine learning. This article revisits the popular Langevin Monte Carlo (LMC) sampling algorithms and provides a non-asymptotic error analysis in $\mathcal{W}_2$-distance in a non-convex setting. In particular, we prove an error bound $O(\sqrt{d} h)$, which guarantees a mixing time $ \tilde{O} (\sqrt{d} \epsilon^{-1})$ to achieve the accuracy tolerance $\epsilon$, under certain log-smooth conditions and the assumption that the target distribution satisfies a log-Sobolev inequality, as opposed to the strongly log-concave condition used in (Li et al., 2019; 2022). This bound matches the best one in the strongly log-concave case and improves upon the best-known convergence rates in non-convex settings. To prove it, we establish a new framework of uniform-in-time convergence for discretizations of SDEs. Distinct from (Li et al., 2019; 2022), we start from the finite-time mean-square fundamental convergence theorem, which combined with uniform-in-time moment bounds of LMC and the exponential ergodicity of SDEs in the non-convex setting gives the desired uniform-in-time convergence. Our framework also applies to the case when the gradient of the potential $U$ is non-globally Lipschitz with superlinear growth, for which modified LMC samplers are proposed and analyzed, with a non-asymptotic error bound in $\mathcal{W}_2$-distance obtained. Numerical experiments corroborate the theoretical analysis.

Lay Summary:

Sampling from complex and high-dimensional probability distributions is a fundamental task with wide-ranging applications in the area of scientific computing, statistics and machine learning. Among classical sampling algorithms, Langevin Monte Carlo (LMC) stands out as a powerful and widely adopted approach. While the theoretical analysis of LMC is well-established for strongly convex target distributions, the error analysis in non-convex settings, more relevant in applications, is rather challenging and is far from being well-understood.

To illustrate it, consider an analogy where the sampling process resembles an explorer navigating a rugged terrain. In a convex landscape, the explorer faces a single dominant peak with a straightforward ascent path. In contrast, non-convex landscapes exhibit multiple isolated peaks separated by deep valleys, where the explorer risks becoming trapped in local regions, potentially overlooking other critical areas of the distribution. This inherent complexity poses substantial challenges to guaranteeing both sampling accuracy and computational efficiency.

In this work, we develop a novel theoretical framework to carry out a non-asymptotic error analysis in a non-convex setting. Our results show that the error bound of LMC is of order $O(\sqrt{d}h)$ in the non-convex setting of log-Sobolev inequality, which matchs the best one in the strongly convex case.

Primary Area: Probabilistic Methods->Monte Carlo and Sampling Methods

Keywords: Langevin Monte Carlo, non-convex setting, optimal dimension-dependence, optimal error bounds in W2-distance

Submission Number: 4727