Expectation Programming: Adapting Probabilistic Programming Systems to Estimate Expectations EfficientlyDownload PDF

Published: 21 Dec 2020, Last Modified: 05 May 2023AABI2020Readers: Everyone
Keywords: Target-Aware Bayesian Inference, Probabilistic Programming, Annealed Importance Sampling
TL;DR: We use ideas from probabilistic programming to automate the estimation of expectations instead of conditional distributions.
Abstract: NOTE: A full paper version of this abstract has been accepted to UAI 2022. Building on ideas from probabilistic programming, we introduce the concept of an expectation programming framework (EPF) that automates the calculation of expectations. Analogous to a probabilistic program, an expectation program is comprised of a mix of probabilistic constructs and deterministic calculations, between which a conditional distribution over internal variables and outputs is defined. However, the focus of the inference engine in an EPF is to directly calculate the expectation of the program return values, rather than this conditional distribution. This is made possible by exploiting recent advancements in target-aware Bayesian inference, through which we can tailor our inference engines to this expectation estimation, providing the potential for substantial improvements over the standard probabilistic programming pipeline. We realize a particular instantiation of our EPF concept by extending the probabilistic programming language Turing with a new @expectation macro that uses a series of program transformations to automatically run target--aware inference. We show that this leads to significant empirical gains in estimation performance compared to conventional use of Turing on two example problems.
1 Reply

Loading