Variational Bayes Made Easy

Mohammad Emtiyaz Khan

Keywords: variational bayes, exponential family, graphical models, expectation maximization

TL;DR: We give a 3-step recipe to simplify the derivation process of variational-Bayes.

Abstract: Variational Bayes is a popular method for approximate inference but its derivation can be cumbersome. To simplify the process, we give a 3-step recipe to identify the posterior form by explicitly looking for linearity with respect to expectations of well-known distributions. We can then directly write the update by simply ``reading-off'' the terms in front of those expectations. The recipe makes the derivation easier, faster, shorter, and more general.