Reproducible sampling from intractable distributions with Pigeons.jl

Miguel Biron-Lattes, Nikola Surjanovic, Paul Tiede, Saifuddin Syed, Trevor Campbell, Alexandre Bouchard-Côté

Keywords: Distributed computation, Bayesian inference, Markov chain Monte Carlo, Parallelism Invariance, Message Passing Interface, Multi-Threading

Abstract:

Sampling from complicated probability distributions---such as multimodal posteriors arising in Bayesian inference and high-dimensional
distributions in statistical mechanics---is a challenging problem. Pigeons.jl is a Julia package that implements Parallel Tempering, a technique that leverages parallelism to improve the effectiveness of Markov chain Monte Carlo algorithms. Pigeons.jl provides a simple user interface to perform such computations single-threaded, multi-threaded, and/or distributed over thousands of MPI-communicating machines. In this paper, we discuss a feature of Pigeons.jl that we call strong parallelism invariance (SPI): a guarantee that the output for a given seed is identical irrespective of the number of threads and processes. This feature is crucial for scientific reproducibility and software validation. We describe the key features of Pigeons.jl that enable a distributed and randomized algorithm satisfying SPI. Finally, we briefly discuss
some recent developments in Pigeons.jl.

Submission Number: 37