JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models

Stefan T. Radev, Marvin Schmitt, Valentin Pratz, Umberto Picchini, Ullrich Koethe, Paul Buerkner

Keywords: simulation-based-inference, surrogate models, Bayesian inference, invertible neural networks, amortized inference

TL;DR: This work proposes a deep learning method for simulatenously approximating intractable likelihood functions and posterior densities arising in surrogate modeling and simulation-based inference.

Abstract: This work proposes ``jointly amortized neural approximation'' (JANA) of intractable likelihood functions and posterior densities arising in Bayesian surrogate modeling and simulation-based inference. We train three complementary networks in an end-to-end fashion: 1) a summary network to compress individual data points, sets, or time series into informative embedding vectors; 2) a posterior network to learn an amortized approximate posterior; and 3) a likelihood network to learn an amortized approximate likelihood. Their interaction opens a new route to amortized marginal likelihood and posterior predictive estimation -- two important ingredients of Bayesian workflows that are often too expensive for standard methods. We benchmark the fidelity of JANA on a variety of simulation models against state-of-the-art Bayesian methods and propose a powerful and interpretable diagnostic for joint calibration. In addition, we investigate the ability of recurrent likelihood networks to emulate complex time series models without resorting to hand-crafted summary statistics.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/jana-jointly-amortized-neural-approximation/code)