Path Integral Sampler: A Stochastic Control Approach For Sampling

Qinsheng Zhang, Yongxin Chen

Keywords: Sampling, Path Integral, Stochastic Differential Equation, MCMC

Abstract: We present Path Integral Sampler~(PIS), a novel algorithm to draw samples from unnormalized probability density functions. The PIS is built on the Schr\"odinger bridge problem which aims to recover the most likely evolution of a diffusion process given its initial distribution and terminal distribution. The PIS draws samples from the initial distribution and then propagates the samples through the Schr\"odinger bridge to reach the terminal distribution. Applying the Girsanov theorem, with a simple prior diffusion, we formulate the PIS as a stochastic optimal control problem whose running cost is the control energy and terminal cost is chosen according to the target distribution. By modeling the control as a neural network, we establish a sampling algorithm that can be trained end-to-end. We provide theoretical justification of the sampling quality of PIS in terms of Wasserstein distance when sub-optimal control is used. Moreover, the path integrals theory is used to compute importance weights of the samples to compensate for the bias induced by the sub-optimality of the controller and the time-discretization. We experimentally demonstrate the advantages of PIS compared with other start-of-the-art sampling methods on a variety of tasks.

One-sentence Summary: We present Path Integral Sampler~(PIS), an efficient algorithm to draw samples from unnormalized probability density functions.

Community Implementations: [ 4 code implementations](https://www.catalyzex.com/paper/arxiv:2111.15141/code)