Robust Learning of Tractable Probabilistic Models

Rohith Peddi, Tahrima Rahman, Vibhav Giridhar Gogate

Keywords: Tractable Probabilistic Models, Robust Maximum Likelihood, Probabilistic Circuits, Sum-Product Networks, Cutset Networks

TL;DR: A new polynomial time algorithm for robust maximum likelihood parameter estimation in tractable probabilistic models

Abstract: Tractable probabilistic models (TPMs) compactly represent a joint probability distribution over a large number of random variables and admit polynomial time computation of (1) exact likelihoods; (2) marginal probability distributions over a small subset of variables given evidence; and (3) in some cases most probable explanations overall non-observed variables given observations. In this paper, we leverage these tractability properties to solve the robust maximum likelihood parameter estimation task in TPMs under the assumption that a TPM structure and complete training data is provided as input. Specifically, we show that TPMs learned by optimizing the likelihood perform poorly when data is subject to adversarial attacks/noise/perturbations/corruption and we can address this issue by optimizing robust likelihood. To this end, we develop an efficient approach for constructing uncertainty sets that model data corruption in TPMs and derive an efficient gradient-based local search method for learning TPMs that are robust against these uncertainty sets. We empirically demonstrate the efficacy of our proposed approach on a collection of benchmark datasets.