Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Learning Sparse Latent Representations with the Deep Copula Information Bottleneck
Aleksander Wieczorek*, Mario Wieser*, Damian Murezzan, Volker Roth
Feb 15, 2018 (modified: Mar 22, 2018)ICLR 2018 Conference Blind Submissionreaders: everyoneShow Bibtex
Abstract:Deep latent variable models are powerful tools for representation learning. In this paper, we adopt the deep information bottleneck model, identify its shortcomings and propose a model that circumvents them. To this end, we apply a copula transformation which, by restoring the invariance properties of the information bottleneck method, leads to disentanglement of the features in the latent space. Building on that, we show how this transformation translates to sparsity of the latent space in the new model. We evaluate our method on artificial and real data.
TL;DR:We apply the copula transformation to the Deep Information Bottleneck which leads to restored invariance properties and a disentangled latent space with superior predictive capabilities.
Keywords:Information Bottleneck, Deep Information Bottleneck, Deep Variational Information Bottleneck, Variational Autoencoder, Sparsity, Disentanglement, Interpretability, Copula, Mutual Information
Enter your feedback below and we'll get back to you as soon as possible.