Maximum Likelihood Learning of Energy-Based Models for Simulation-Based Inference

Pierre Glaser, Michael Arbel, Arnaud Doucet, Arthur Gretton

Keywords: Simulation Based Inference, Energy Based Models, Maximum Likelihood

TL;DR: We introduce two Synthetic Likelihood methods for Simulation-Based Inference using Conditional Energy-Based Models

Abstract:

We introduce two Synthetic Likelihood methods for Simulation-Based Inference (SBI), to conduct either amortized or targeted inference from experimental observations when a high-fidelity simulator is available. Both methods learn a Conditional Energy-Based Model (EBM) of the likelihood using synthetic data generated by the simulator, conditioned on parameters drawn from a proposal distribution. The learned likelihood can then be combined with any prior to obtain a posterior estimate, from which samples can be drawn using MCMC.
Our methods uniquely combine a flexible Energy-Based Model and the minimization of a KL loss: this is in contrast to other synthetic likelihood methods, which either rely on normalizing flows, or minimize score-based objectives; choices that come with known pitfalls. Our first method, Amortized Unnormalized Neural Likelihood Estimation (AUNLE), introduces a tilting trick during training that allows to perform inference using efficient MCMC techniques. Our second method, Sequential UNLE (SUNLE), employs a doubly intractable approach in order to re-use simulation data and improve posterior accuracy for a specific observation.
We demonstrate the properties of both methods on a range of synthetic datasets, and apply it to a neuroscience model of the pyloric network in the crab, matching the performance of other synthetic likelihood methods at a fraction of the simulation budget.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Machine Learning for Sciences (eg biology, physics, health sciences, social sciences, climate/sustainability )