Learning Robust Representations via Multi-View Information Bottleneck


Sep 25, 2019 ICLR 2020 Conference Blind Submission readers: everyone Show Bibtex
  • TL;DR: We extend the information bottleneck method to the unsupervised multiview setting and show state of the art results on standard datasets
  • Abstract: The information bottleneck method provides an information-theoretic view of representation learning. The original formulation, however, can only be applied in the supervised setting where task-specific labels are available at learning time. We extend this method to the unsupervised setting, by taking advantage of multi-view data, which provides two views of the same underlying entity. A theoretical analysis leads to the definition of a new multi-view model which produces state-of-the-art results on two standard multi-view datasets, Sketchy and MIR-Flickr. We also extend our theory to the single-view setting by taking advantage of standard data augmentation techniques, empirically showing better generalization capabilities when compared to traditional unsupervised approaches.
  • Code: https://github.com/anonymous003784/Multi-View-Information-Bottleneck
  • Keywords: Information Bottleneck, Multi-View Learning, Representation Learning, Information Theory
0 Replies