On Assimilating Learned Views in Contrastive Learning

Ligong Han, Seungwook Han, Shivchander Sudalairaj, Charlotte Loh, Rumen Dangovski, Pulkit Agrawal, Dimitris N. Metaxas, Leonid Karlinsky, Tsui-Wei Weng, Akash Srivastava

Keywords: Contrastive Learning, Self-Supervised Learning, Generative Models, Mutual Information

Abstract: Transformations based on domain expertise (expert transformations), such as random-resized-crop and color-jitter, have proven critical to the success of contrastive learning techniques such as SimCLR. Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned. However, for imagery data, so far none of these view generation methods has been able to outperform expert transformations. In this work, we tackle a different question: instead of replacing expert transformations with generated views, can we constructively assimilate generated views with expert transformations? We answer this question in the affirmative. To do so, we first propose an information-theoretic framework for designing view generation based on the analysis of Tian et al 2020b on what makes a "good" view in contrastive learning. Then, we present two simple yet effective assimilation methods that together with our view generation mechanisms improve the state-of-the-art by up to approximately 3.5% on four different datasets. Importantly, we conduct a detailed empirical study that systematically analyzes a range of view generation and assimilation methods and provides a holistic picture of the efficacy of learned views in contrastive representation learning.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Unsupervised and Self-supervised learning