Model Exploration through Marginal Likelihood Entropy Maximisation

Daniel Jarne Ornia, Joel Dyer, Nicholas George Bishop, Ani Calinescu, Michael J. Wooldridge

Keywords: maximum entropy, model exploration, simulation model

Abstract: Simulation modelling offers a flexible approach to constructing high-fidelity synthetic representations of real-world complex systems. The appeal of such models often lies in their ability to facilitate scenario exploration: exploring the different possible futures that could manifest in a complex system. Using simulators for this purpose requires efficient procedures for exploring the range of possible behaviours the simulator can produce. In this paper, we propose and investigate a method to efficiently explore, in an end-to-end parametric manner, the different behaviours that can arise from stochastic, differentiable black-box simulators. Our approach entails maximising the entropy of the marginal likelihood function induced by a trainable proposal distribution over the model's parameter space, computed using direct entropy estimators of the simulated outputs. The method does not require the simulators to have tractable likelihood functions, does not entail building entropy surrogates or instantiating multiple different models, and can be easily parallelised. We provide a proof-of-concept demonstration of the effectiveness of our proposed method on an epidemic simulator commonly used in the literature.

Submission Number: 13