Encoding Negative Dependencies in Probabilistic Circuits

Aleksanteri Mikulus Sladek, Martin Trapp, Arno Solin

Keywords: probabilistic circuit, negative dependencies, squared neural families

Abstract: Tractability is considered key to trustworthy decision-making under uncertainty, but it often comes at the expense of the ability to represent large families of probability distributions. Probabilistic circuits promise to remedy this by representing tractable yet expressive probabilistic models through hierarchical compositions of tractable distributions subject to certain structural and parameter constraints. A common parameter constraint enforced in these models is non-negativity of the weights, which has been shown by prior work to potentially hinder their expressive efficiency. In this work, we propose allowing for negative weights in probabilistic circuits by loosening the non-negativity constraint to a positive semidefinite constraint. We empirically show that probabilistic circuits with positive semidefinite parameterized nodes have increased expressive efficiency, whilst retaining tractability, and empirically outperform circuits with non-negative weight constraints.

Submission Number: 14