Noisy Natural Gradient as Variational Inference

Guodong Zhang, Shengyang Sun, David K. Duvenaud, Roger B. Grosse

Abstract: Variational Bayesian neural nets combine the flexibility of deep learning with Bayesian uncertainty estimation. Unfortunately, there is a tradeoff between cheap but simple variational families (e.g...