SpatFormer: simulation-based inference with transformers for spatial statistics

Herman Tesso, Ayush Bharti, Elizaveta Semenova

Keywords: simulation-based inference, transformers, diffusion models, spatial statistics

Abstract: Gaussian Process (GP) priors are widely used in spatial statistical models but suffer from cubic computational complexity in Markov Chain Monte Carlo (MCMC), limiting scalability. We propose a simulation-based inference (SBI) method using a transformer-enhanced diffusion model tailored for spatial models with latent GP priors. By leveraging transformers for sequence modeling and probabilistic diffusion for inference, our approach enables efficient, amortized Bayesian inference. Unlike traditional MCMC, it avoids repeated costly computations, allowing rapid exploration of spatial conditional distributions. Simulation experiments on one- and two-dimensional models demonstrate superior scalability compared to GP-based MCMC, making our method well-suited for large-scale applications in spatial epidemiology and other domains. The accompanying code is available at https://github.com/hermanFTT/SpatFormer.

Submission Number: 35