IL-Strudel : Independence-Based Learning of Structured-Decomposable Probabilistic Circuit Ensembles

Shreyas Kowshik, Yitao Liang, Guy Van den Broeck

Keywords: Probabilistic Circuits, Structured Decomposability, Mixture Models

TL;DR: Developing a strong initialization strategy for ensemble learning of structured-decomposable probabilistic-circuits by enforcing context-specific-independences at component roots

Abstract: Probabilistic Sentential Decision Diagrams (PSDD) are a class of highly tractable structured-decomposable probabilistic circuits. They provide for closed-form parameter learning and capture structure from data in the form of context-specific independences. When learning mixture models of such circuits, the Soft EM algorithm is used for parameter learning which is highly sensitive to initialization. These independence properties can be a valuable source of prior information for component initialization. We hypothesize that if each component structure can capture independences on disjoint sub-supports of the data, the overall ensemble can get a boost in performance. Through this paper, we first develop a framework for connecting these independences to likelihood-based structure evaluation. Using this framework, we propose a novel algorithm to learn a stronger mixture model by providing an initialization strategy to enforce context-specific-independences at the component root levels. Our experimental results validate our approach as it beats previous approaches on 14 out of 20 datasets.