Properties and limitations of geometric tempering for gradient flow dynamics

Francesca Romana Crucinio, Sahani Pathiraja

Abstract:

We consider the problem of sampling from a probability distribution $\pi$. It is well known that this can be written as an optimisation problem over the space of probability distributions in which we aim to minimise the Kullback--Leibler divergence from $\pi$.
We consider the effect of replacing $\pi$ with a sequence of moving targets $(\pi_t)_{t\ge0}$ defined via geometric tempering on the Wasserstein and Fisher--Rao gradient flows.
We show that convergence occurs exponentially in continuous time, providing novel bounds in both cases. We also consider popular time discretisations and explore their convergence properties.
We show that in the Fisher--Rao case, replacing the target distribution with a geometric mixture of initial and target distribution never leads to a convergence speed up both in continuous time and in discrete time. Finally, we explore the gradient flow structure of tempered dynamics and derive novel adaptive tempering schedules.

Submission Type: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=DfiHvr2Z8X&noteId=Jj4tX8hPoO

Changes Since Last Submission: Fixed URL and date/time.

Code: https://github.com/FrancescaCrucinio/SMC-WFR/tree/main/TemperedDynamics

Assigned Action Editor: ~Trevor_Campbell1

Submission Number: 7462