Sampling via Controlled Stochastic Dynamical Systems

Benjamin Zhang, Tuhin Sahai, Youssef Marzouk

Keywords: Stochastic differential equations, sampling, stochastic optimal control, Schrodinger bridge

TL;DR: Controlled SDEs can sample from target probability distributions.

Abstract: We present a general framework for constructing controlled stochastic dynamical systems that exactly sample from a class of probability distributions with Gaussian tails. Given a target distribution and a reference stochastic differential equation (SDE), the Doob $h$-transform produces a controlled stochastic process whose marginal at a finite time $T$ will be equal to the target distribution. Our method constructs a reference linear SDE and uses the eigenfunctions of its associated Markov operator to approximate the Doob $h$-transform. The control is approximated by projecting the ratio between the target density and the reference system’s time $T$ marginal onto the span of a finite set of eigenfunctions. This projection is performed by minimizing the Kullback-Leibler (KL) divergence from the marginal produced by the approximate control to the true target distribution. In practice, the method lacks robustness due to the high sensitivity to the algorithm's parameters.

Category: Stuck paper: I hope to get ideas in this workshop that help me unstuck and improve this paper