Discriminative bias for classification with robust probabilistic models

Laura Isabel Galindez Olascoaga, Wannes Meert, Nimish Shah, Guy Van den Broeck, Marian Verhelst

TL;DR: This paper presents a probabilistic model learning strategy, that enforces the discriminative relationship between class and feature variables in a generative model to achieve better classification accuracy while remaining robust to missing features.

Keywords: Probabilistic circuits, Probabilistic Sentential Decision Diagrams, Generative learning, Missing features, Probabilistic inference

Abstract: Probabilistic Sentential Decision Diagrams (PSDD) are a type of probabilistic circuit that can be learned incrementally from data by iteratively optimizing the log-likelihood of the induced distribution. As generative models, PSDDs remain robust against missing features but are often outperformed by discriminatively trained models in classification tasks. We consider the recently published D-LearnPSDD learner, which explicitly encodes the discriminative relation between class and feature variables to improve classification performance, while still reaping the robustness benefits of generative learning. In this work we further generalize the original contribution's theorems to consider multi-value classification scenarios, and we discuss the implementation techniques suitable for those scenarios.