Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Deep Variational Information Bottleneck
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, Kevin Murphy
Nov 04, 2016 (modified: Feb 22, 2017)ICLR 2017 conference submissionreaders: everyone
Abstract:We present a variational approximation to the information bottleneck of Tishby et al. (1999). This variational approach allows us to parameterize the information bottleneck model using a neural network and leverage the reparameterization trick for efficient training. We call this method “Deep Variational Information Bottleneck”, or Deep VIB. We show that models trained with the VIB objective outperform those that are trained with other forms of regularization, in terms of generalization performance and robustness to adversarial attack.
TL;DR:Applying the information bottleneck to deep networks using the variational lower bound and reparameterization trick.
Keywords:Theory, Computer vision, Deep learning, Supervised Learning
Enter your feedback below and we'll get back to you as soon as possible.