When VMP Meets CEP: An Algorithmic Equivalence Under Mild Conditions

Siyuan Li, Shikai Fang, Lei Cheng, Yik-Chung WU, Sergios Theodoridis

Abstract: Approximate Bayesian inference (ABI) methods have become indispensable tools in modern machine learning and statistics for approximating intractable posterior distributions. Despite extensive studies, the theoretical connections among different ABI methods have remained relatively unexplored. This paper establishes an algorithmic equivalence between two widely employed ABI techniques, namely variational message passing (VMP) and conditional expectation propagation (CEP). Through rigorous mathematical analysis, we demonstrate that these two approaches, despite originating from different perspectives (variational inference and expectation propagation, respectively), yield the same update equations under mild conditions, from both optimization and graphical model viewpoints. As a direct consequence, we establish a convergence guarantee for CEP and show that VMP-derived algorithms can inherit streaming variants without additional derivation effort. To validate our theoretical findings, we apply both VMP and CEP to Bayesian tensor decomposition and verify that they produce identical updates, demonstrating how the equivalence provides a principled route to a streaming variant.

Submission Type: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=alODfvLNuP&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)

Changes Since Last Submission: Revised the title, recalibrated claims to match the scope of the results, and fixed notation and terminology issues.

Assigned Action Editor: ~Yingzhen_Li1

Submission Number: 6740