Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization

Kyurae Kim, Zuheng Xu, Jacob R. Gardner, Trevor Campbell

TL;DR: We propose a gradient- and tuning-free online scheme for tuning the proposal kernels in sequential Monte Carlo samplers.

Abstract: The performance of sequential Monte Carlo (SMC) samplers heavily depends on the tuning of the Markov kernels used in the path proposal. For SMC samplers with unadjusted Markov kernels, standard tuning objectives, such as the Metropolis-Hastings acceptance rate or the expected-squared jump distance, are no longer applicable. While stochastic gradient-based end-to-end optimization algorithms have been explored for tuning SMC samplers, they often incur excessive training costs, even for tuning just the kernel step sizes. In this work, we propose a general adaptation framework for tuning the Markov kernels in SMC samplers by minimizing the incremental Kullback-Leibler (KL) divergence between the proposal and target paths. For step size tuning, we provide a gradient- and tuning-free algorithm that is generally applicable for kernels such as Langevin Monte Carlo (LMC). We further demonstrate the utility of our approach by providing a tailored scheme for tuning kinetic LMC used in SMC samplers. Our implementations are able to obtain a full schedule of tuned parameters at the cost of a few vanilla SMC runs, which is a fraction of gradient-based approaches.

Lay Summary:

Sequential Monte Carlo (SMC) is a popular algorithm in statistics, physics, and machine learning for numerically evaluating high-dimensional integrals over probability distributions. It is particularly relevant for the purpose of comparing scientific hypotheses within the Bayesian framework, a process known as Bayesian model comparison.

In practice, however, SMC tends to be difficult to use due to the abundance of tunable parameters. Furthermore, SMC is an iterative algorithm, where each step comes with its own set of tunable parameters. This is particularly problematic for certain types of SMC samplers that internally use "unadjusted MCMC kernels." Previous automatic tuning approaches had to rely on end-to-end optimization, which is particularly expensive. These methods attempt to tune all of the parameters at once by performing stochastic gradient descent. As a result, they requires running SMC many times and involve higher-order derivatives in the process. In this work, we propose an approach to tuning SMC samplers that is much cheaper and does not involve higher-order derivatives. The key idea is that we break up the tuning problem into multiple subproblems, one for each step of SMC. Each subproblem can be easily solved online without having to involve derivatives.

Our proposed scheme significantly reduces the computational cost of using SMC samplers. On some problems, it even improves their statistical accuracy compared to end-to-end optimization approaches.

Link To Code: https://github.com/Red-Portal/ControlledSMC.jl/tree/v0.0.4

Primary Area: Probabilistic Methods->Monte Carlo and Sampling Methods

Keywords: sequantial Monte Carlo, annealed importance sampling, Markov chain Monte Carlo, Bayesian inference

Submission Number: 2672