Negative Weights in Hinge-Loss Markov Random Fields

Charles Andrew Dickens, Eriq Augustine, Connor Pryor, Lise Getoor

Abstract: Hinge-loss Markov random fields (HL-MRF) are a class of probabilistic graphical models with density functions that admit tractable MAP inference. When paired with an expressive modeling framework, HL-MRFs are powerful tools for performing structured prediction. One such framework, probabilistic soft logic (PSL), uses weighted first-order logical statements to incorporate domain knowledge and constraints into the HL-MRF structure. Traditionally, PSL restricts weights to be non-negative to ensure MAP inference remains tractable, but this limits the types of relations PSL models can represent. We propose three novel approaches to extending PSL's expressivity to support negative weights. Notably, we propose the use of Gödel logic for defining potentials from negatively weighted rules. This method improves upon prior work on this topic by preserving both the convexity and scale of the MAP inference problem. Moreover, we show where our new methods and two approaches from prior work overlap and where they most differ. All methods are implemented in PSL, and we introduce a tunable synthetic dataset designed to empirically compare the performance of predictions.