Stochastic complexity of variational Bayesian hidden Markov models

Tikara Hosino, Kazuho Watanabe, Sumio Watanabe

Abstract: Variational Bayesian learning was proposed as the approximation method of Bayesian learning. Inspite of efficiency and experimental good performance, their mathematical property has not yet been clarified. In this paper we analyze variational Bayesian hidden Markov models which include the true one thus the models are non-identifiable. We derive their asymptotic stochastic complexity. It is shown that, in some prior condition, the stochastic complexity is much smaller than those of identifiable models.