metabeta - A fast neural model for Bayesian mixed-effects regression

ICLR 2026 Conference Submission21661 Authors

Keywords: neural posterior estimation, mixed-effects regression, approximate bayesian computation

TL;DR: This paper showcases a pretrained neural model that efficiently approximates posterior inference for mixed-effects regression.

Abstract: Hierarchical data with multiple observations per group is ubiquitous in empirical sciences and is often analyzed using mixed-effects regression. In such models, Bayesian inference gives an estimate of uncertainty but is analytically intractable and requires costly approximation using Markov Chain Monte Carlo (MCMC) methods. Neural posterior estimation shifts the bulk of computation from inference time to pre-training time, amortizing over simulated datasets with known ground truth targets. We propose metabeta, a transformer-based neural network model for Bayesian mixed-effects regression. Using simulated and real data, we show that it reaches stable and comparable performance to MCMC-based parameter estimation at a fraction of the usually required time.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 21661