Are gradients worth the effort? Comparing automatic differentiation and simulation-based inference for agent-based models

Timothy James Hitge, Arnau Quera-Bofarull, Elizaveta Semenova

Keywords: Automatic differentiation, Simulation-based inference, Agent-based models, Generalized variational inference

Abstract: Agent-based models (ABMs) are flexible tools for simulating complex systems, but their calibration is difficult because their likelihoods are intractable and simulations are expensive. Two modern approaches tackle this challenge: automatic differentiation (AD), which makes simulators differentiable to enable gradient-based optimisation, and simulation-based inference (SBI), which learns approximate posteriors from simulated data without changing the simulator. Despite their growing use for inferring belief distributions over model parameters, these methods have not been directly compared for ABMs. We present an empirical study comparing AD-based variational inference and SBI on a spatial SIRS model. We evaluate the methods based on the trade-offs they present between predictive accuracy, sample efficiency, and implementation complexity. While our results suggest that SBI is preferable for ABMs with low-dimensional parameter spaces, they also highlight the need for future research, which we outline in our discussion. Code for reproducibility is available at https://github.com/SteamedGit/ad_vs_sbi_workshop.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 13