Functional Tensors for Probabilistic Programming

Fritz Obermeyer, Eli Bingham, Martin Jankowiak, Du Phan, Jonathan Chen

TL;DR: We generalize tensors to multivariate functions and distributions.

Abstract: It is a significant challenge to design probabilistic programming systems that can accommodate a wide variety of inference strategies within a unified framework. Noting that the versatility of modern automatic differentiation frameworks is based in large part on the unifying concept of tensors, we describe a software abstraction, functional tensors, that captures many of the benefits of tensors, while also being able to describe continuous probability distributions. We demonstrate the versatility of functional tensors by integrating them into the modeling frontend and inference backend of the Pyro probabilistic programming language. As an example application, we perform approximate inference on a switching linear dynamical system.

Keywords: probabilistic programming, symbolic algebra, approximate inference

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/arxiv:1910.10775/code)