MINE: Mutual Information Neural Estimation

Mohamed Ishmael Belghazi, Sai Rajeswar, Aristide Baratin, Devon Hjelm, Aaron Courville

Abstract: This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size. MINE is back-propable and we prove that it is strongly consistent. We illustrate a handful of applications in which MINE is succesfully applied to enhance the property of generative models in both unsupervised and supervised settings. We apply our framework to estimate the information bottleneck, and apply it in tasks related to supervised classification problems. Our results demonstrate substantial added flexibility and improvement in these settings.

TL;DR: A scalable in sample size and dimensions mutual information estimator.

Keywords: Deep Learning, Neural Networks, Information Theory, Generative models, GAN, Adversarial

Community Implementations: [ 17 code implementations](https://www.catalyzex.com/paper/mine-mutual-information-neural-estimation/code)