Learning Likelihood-Free Reference Priors

Nicholas George Bishop, Joel Dyer, Daniel Jarne Ornia, Ani Calinescu, Michael J. Wooldridge

Keywords: SBI, Bayesian Inference, Objective Priors

TL;DR: We propose a new method for learning reference priors likelihood-free!

Abstract:

Simulation modeling offers a flexible approach to constructing high-fidelity synthetic representations of complex real-world systems.
However, the increased complexity of such models introduces additional complications when carrying out statistical inference procedures.

This has motivated a large and growing literature on \textit{likelihood-free} or \textit{simulation-based} inference methods, which approximate (e.g., Bayesian) inference without assuming access to the simulator's intractable likelihood function.

A hitherto neglected problem in the simulation-based Bayesian inference literature is the challenge of constructing uninformative \textit{reference priors} for complex simulation models.

Such priors maximise an expected Kullback-Leibler distance from the prior to the posterior, thereby influencing posterior inferences minimally and enabling an ``objective'' approach to Bayesian inference that do not necessitate the incorporation of strong subjective prior beliefs.

In this paper, we propose and test a selection of likelihood-free methods for learning reference priors for simulation models, using variational approximations and a variety of mutual information estimators.

Our experiments demonstrate that good approximations to reference priors for simulation models are in this way attainable, providing a first step towards the development of likelihood-free objective Bayesian inference procedures.

Submission Number: 36