A Proximal Algorithm for Sampling

Published: 03 Jul 2023, Last Modified: 03 Jul 2023Accepted by TMLREveryoneRevisionsBibTeX
Abstract: We study sampling problems associated with potentials that lack smoothness. The potentials can be either convex or non-convex. Departing from the standard smooth setting, the potentials are only assumed to be weakly smooth or non-smooth, or the summation of multiple such functions. We develop a sampling algorithm that resembles proximal algorithms in optimization for this challenging sampling task. Our algorithm is based on a special case of Gibbs sampling known as the alternating sampling framework (ASF). The key contribution of this work is a practical realization of the ASF based on rejection sampling for both non-convex and convex potentials that are not necessarily smooth. In almost all the cases of sampling considered in this work, our proximal sampling algorithm achieves a better complexity than all existing methods.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: See red marks. Add new numerical results on MALA, Figures 4 and 5.
Code: https://www.dropbox.com/sh/25ku9is6g7dxikv/AADcLBJipPLD30T1XjOf1h1_a?dl=0
Assigned Action Editor: ~Peter_Richtarik1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 1049