ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian ModelsDownload PDF

29 Sept 2021, 00:32 (edited 07 Mar 2022)ICLR 2022 PosterReaders: Everyone
  • Keywords: Bayesian inference, Hierarchical Bayesian Models, structured Variational Inference, Simulation Based Inference, Inference amortization, Neuroimaging
  • Abstract: Frequently, population studies feature pyramidally-organized data represented using Hierarchical Bayesian Models (HBM) enriched with plates. These models can become prohibitively large in settings such as neuroimaging, where a sample is composed of a functional MRI signal measured on 300 brain locations, across 4 measurement sessions, and 30 subjects, resulting in around 1 million latent parameters. Such high dimensionality hampers the usage of modern, expressive flow-based techniques. To infer parameter posterior distributions in this challenging class of problems, we designed a novel methodology that automatically produces a variational family dual to a target HBM. This variational family, represented as a neural network, consists in the combination of an attention-based hierarchical encoder feeding summary statistics to a set of normalizing flows. Our automatically-derived neural network exploits exchangeability in the plate-enriched HBM and factorizes its parameter space. The resulting architecture reduces by orders of magnitude its parameterization with respect to that of a typical flow-based representation, while maintaining expressivity. Our method performs inference on the specified HBM in an amortized setup: once trained, it can readily be applied to a new data sample to compute the parameters' full posterior. We demonstrate the capability and scalability of our method on simulated data, as well as a challenging high-dimensional brain parcellation experiment. We also open up several questions that lie at the intersection between normalizing flows, SBI, structured Variational Inference, and inference amortization.
  • One-sentence Summary: We automatically derive a variational family dual to a plate-enriched Hierarchical Bayesian Network and perform amortized inference.
  • Supplementary Material: zip
22 Replies