Fixing the Bethe Approximation: How Structural Modifications in a Graph Improve Belief Propagation

Harald Leisenberger, Franz Pernkopf, Christian Knoll

Keywords: Belief Propagation, Bethe Free Energy Approximation, Graphical Models

TL;DR: We demonstrate and analyze the beneficial effect of removing edges from a graph on the Bethe approximation and belief propagation.

Abstract: Belief propagation is an iterative method for inference in probabilistic graphical models. Its well-known relationship to a classical concept from statistical physics, the Bethe free energy, puts it on a solid theoretical foundation. If belief propagation fails to approximate the marginals, then this is often due to a failure of the Bethe approximation. In this work, we show how modifications in a graphical model can be a great remedy for fixing the Bethe approximation. Specifically, we analyze how the removal of edges influences and improves belief propagation, and demonstrate that this positive effect is particularly distinct for dense graphs.