Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Asymptotic Free Energy Of Variational Bayesian Deep Learning
Hiroshi Wakimori, Tikara Hosino
Feb 09, 2018 (modified: Jun 04, 2018)ICLR 2018 Workshop Submissionreaders: everyoneShow Bibtex
Abstract:The Bayesian deep learning is promising for its theoretical foundation. Especially, it was probed that free energy can asymptotically identify the structure of the true distribution consistently. In this paper, we derive the asymptotic expected variational free energy in the case of Gaussian trial posterior. The result shows that the variance of the posterior reflects the relative structure of the true distribution and the learning model. This result clarifies the theoretical insights of model selection and model distillation in variational approximation of Bayesian methods.
Keywords:variational inference, free energy, deep learning, model selection, model distillation
TL;DR:We derive the asymptotic expected variational free energy of the Bayesian deep learning in the case of Gaussian trial posterior.
Enter your feedback below and we'll get back to you as soon as possible.