A COMPASS to Bayesian Model Comparison in Simulator-Based Settings

Berkay Günes, Tobias Buck

Keywords: simulation-based inference, model comparison, transformer, diffusion models, statistics

TL;DR: This paper introduces compass, a framework that leverages a single, maskable Diffusion Transformer to unify posterior and likelihood estimation, enabling efficient and interpretable Bayesian Model Comparison for scientific simulators.

Abstract:

Bayesian Model Comparison (BMC) is a cornerstone of scientific discovery, yet it remains a formidable challenge for complex, simulation-based models where likelihoods are intractable. Existing Simulation-Based Inference (SBI) methods primarily focus on parameter inference and can be computationally prohibitive to adapt for comparing multiple model hypotheses. While recent works have introduced highly flexible and powerful inference methods, a dedicated framework for robust and scalable BMC is still lacking.
We introduce \texttt{COMPASS}, a novel framework that leverages a conditional diffusion transformer to create an efficient, end-to-end pipeline specifically for BMC. By strategically masking inputs, \texttt{COMPASS} uses a single, flexibly conditioned model per hypothesis to perform posterior estimation for parameter inference and likelihood estimation for model ranking. It incorporates a principled method for jointly inferring shared parameters from multiple observations, leading to a highly robust estimate of the maximized likelihood for model comparison. We demonstrate \texttt{COMPASS} on two benchmark tasks and a challenging real-world astrophysics application, showing that it correctly identifies the ground-truth data-generating model and provides robust parameter constraints, even under model misspecification. Furthermore, we show that the model's internal attention mechanism is interpretable, providing novel scientific insights into the learned physical relationships that drive model selection. Our work provides a powerful, general-purpose tool for scientific discovery. The code is publicly available at https://anonymous.4open.science/r/COMPASS-6CC6.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 12105