Learning Cutset Networks by Integrating Data and Noisy, Local Estimates

Shasha Jin, Vasundhara Komaragiri, Tahrima Rahman, Vibhav Giridhar Gogate

Keywords: Tractable Probabilistic Models, Parameter Learning, Inconsistent Data, Cutset Networks

TL;DR: A novel method for learning tractable probabilistic models from local, potentially inconsistent data

Abstract: We consider the following parameter learning task in cutset networks (CNs): given (1) fully observed data, (2) a large number of marginal probability distributions, each defined over a small subset of variables, and (3) a CN structure, find a setting of parameters such that the resulting CN efficiently integrates the statistical information present in both the data and marginal distributions. In many real- world applications, the marginal distributions are either available from experts or via external processes and are typically inconsistent in that they do not come from the same joint probability distribution. In order to filter the inconsistency, we propose to approximate the learning objective us- ing a convex combination of two quantities: one that enforces closeness via KL divergence to the marginal distributions and another that enforces closeness to a CN learned from data. We develop a gradient-based algorithm for minimizing the above objective and show that although the gradients are NP-hard to compute on Bayesian and Markov net- works, they can be efficiently computed over CNs yielding a polynomial time algorithm with convergence guarantees. We show via experiments that our approach yields tractable models that are significantly superior to the ones learned from data alone even when the marginal distributions exhibit a high degree of inconsistency.