Analytically Tractable Inference in Deep Neural NetworksDownload PDF

Published: 25 Jul 2021, Last Modified: 22 Oct 2023TPM 2021Readers: Everyone
Keywords: Bayesian Neural Networks, Tractable Inference, Analytical Inference, Convolutional Neural Networks, Generative Neural Networks
TL;DR: Tractable Approximate Bayesian Inference; Apply to CNN and GAN
Abstract: Since its inception, deep learning has been overwhelmingly reliant on backpropagation and gradient-based optimization algorithms in order to learn weight and bias parameter values. The Tractable Approximate Gaussian Inference (TAGI) algorithm was shown to be a viable and scalable alternative to backpropagation for shallow fully-connected neural networks. In this paper, we are demonstrating how TAGI matches or exceeds the performance of backpropagation, for training classic deep neural network architectures. Although TAGI's computational efficiency is still below that of deterministic approaches relying on backpropagation, it outperforms them on classification tasks and matches their performance for information maximizing generative adversarial networks while using smaller architectures trained with fewer epochs.
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 1 code implementation](https://www.catalyzex.com/paper/arxiv:2103.05461/code)
1 Reply

Loading