- TL;DR: We extend the information bottleneck method to the unsupervised multiview setting and show state of the art results on standard datasets
- Abstract: The information bottleneck method provides an information-theoretic view of representation learning. The original formulation, however, can only be applied in the supervised setting where task-specific labels are available at learning time. We extend this method to the unsupervised setting, by taking advantage of multi-view data, which provides two views of the same underlying entity. A theoretical analysis leads to the definition of a new multi-view model which produces state-of-the-art results on two standard multi-view datasets, Sketchy and MIR-Flickr. We also extend our theory to the single-view setting by taking advantage of standard data augmentation techniques, empirically showing better generalization capabilities when compared to traditional unsupervised approaches.
- Code: https://github.com/anonymous003784/Multi-View-Information-Bottleneck
- Keywords: Information Bottleneck, Multi-View Learning, Representation Learning, Information Theory