Variational Message Passing Neural Network for Maximum-A-Posteriori Inference

Zijun Cui, Hanjing Wang, Tian Gao, Kartik Talamadupula, Qiang Ji

Abstract: Maximum-A-Posteriori (MAP) inference is a fundamental task in probabilistic inference and belief propagation (BP) is a widely used algorithm for MAP inference. Though BP has been applied successfully to many different fields, it offers no performance guarantee and often performs poorly on loopy graphs. To improve the performance on loopy graphs and to scale up to large graphs, we propose a variational message passing neural network (V-MPNN), where we leverage both the power of neural networks in modeling complex functions and the well-established algorithmic theories on variational belief propagation. Instead of relying on a hand-crafted variational assumption, we propose a neural free energy where a general variational distribution is parameterized through a neural network. A message passing neural network is utilized for the minimization of neural free energy. Training of the MPNN is thus guided by neural free energy, without requiring exact MAP configurations as annotations. We empirically demonstrate the effectiveness of the proposed V-MPNN by comparing against both state-of-the-art training-free methods and training-based methods.