A problem with (and fix for) variational Bayesian NMF

Matthew D. Hoffman

Abstract: Probabilistic nonnegative matrix factorization (NMF) models have had great success in audio source separation problems. Bayesian formulations of these models are fit either using Markov chain Monte Carlo or variational inference, with the latter often being preferred for its computational efficiency. However, this computational efficiency comes at a cost; mean-field variational methods cannot represent posterior dependences, and can be quite sensitive to local optima. In this work we examine these issues in Bayesian NMF, and find that they can cause serious problems. Fortunately, we find that these problems can be alleviated by employing the recently proposed structured stochastic mean-field algorithm.