Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
STOCHASTIC GRADIENT LANGEVIN DYNAMICS THAT EXPLOIT NEURAL NETWORK STRUCTURE
Zachary Nado, Jasper Snoek, Roger Grosse, David Duvenaud, Bowen Xu, James Martens
Feb 12, 2018 (modified: Feb 13, 2018)ICLR 2018 Workshop Submissionreaders: everyone
Abstract:Tractable approximate Bayesian inference for deep neural networks remains challenging. Stochastic Gradient Langevin Dynamics (SGLD) offers a tractable approximation to the gold standard of Hamiltonian Monte Carlo. We improve on existing methods for SGLD by incorporating a recently-developed tractable approximation of the Fisher information, known as K-FAC, as a preconditioner.
TL;DR:We use a recent approximation for the Fisher information to improve approximate Bayesian inference for deep neural networks with Langevin Dynamics.
Keywords:monte carlo, Bayesian deep networks
Enter your feedback below and we'll get back to you as soon as possible.