Conditionally Tractable Density Estimation using Neural Networks

Hailiang Dong, Chiradeep Roy, Tahrima Rahman, Vibhav Giridhar Gogate, Nicholas Ruozzi

Keywords: tractable models, cutset networks

TL;DR: A novel (conditionally) tractable model for continuous domains motivated by cutset networks.

Abstract: Tractable models such as cutset networks and sum-product networks (SPNs) have become increasingly popular as they admit polynomial time inference in some cases. Among them, cutset networks, which model the mechanics of Pearl’s cutset conditioning algorithm, demonstrate great scalability and prediction accuracy. Existing research on cutset networks has mainly focused on discrete domains, and the best mechanism to extend cutset networks to continuous domains is unclear. We propose one possible alternative to cutset networks that models the full joint distribution as the product of a general, complex distribution over a small subset of variables and a fully tractable conditional distribution whose parameters are controlled by a neural network. This model admits exact inference when all variables in the general distribution are observed, and although the model is not fully tractable in general, we show that "cutset" sampling can be employed to efficiently generate accurate predictions in practice. We show that our model performs comparably or better than existing competitors on a variety of real datasets.