A Proximal Algorithm for Sampling

Jiaming Liang, Yongxin Chen

Abstract:

We study sampling problems associated with potentials that lack smoothness. The potentials can be either convex or non-convex. Departing from the standard smooth setting, the potentials are only assumed to be weakly smooth or non-smooth, or the summation of multiple such functions. We develop a sampling algorithm that resembles proximal algorithms in optimization for this challenging sampling task. Our algorithm is based on a special case of Gibbs sampling known as the alternating sampling framework (ASF). The key contribution of this work is a practical realization of the ASF based on rejection sampling for both non-convex and convex potentials that are not necessarily smooth. In almost all the cases of sampling considered in this work, our proximal sampling algorithm achieves a better
complexity than all existing methods.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: See red marks. Add new numerical results on MALA, Figures 4 and 5.

Code: https://www.dropbox.com/sh/25ku9is6g7dxikv/AADcLBJipPLD30T1XjOf1h1_a?dl=0

Assigned Action Editor: ~Peter_Richtarik1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 1049