GPS-SSL: Guided Positive Sampling to Inject Prior into Self-Supervised Learning
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: self-supervised learning, computer vision, data augmentation, real-world datasets
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: In this paper, we propose Guided Positive Sampling Self-Supervised Learning (GPS-SSL), a general method to embed a priori knowledge into Self-Supervised Learning (SSL) positive samples selection. Current SSL methods leverage Data-Augmentations (DA) for generating positive samples and their performance heavily relies on the chosen set of DA. However, designing optimal DA given a target dataset requires domain knowledge regarding that dataset and can be costly to search and find. Our method designs a metric space where distances better align with semantic relationship thus enabling nearest neighbor sampling to provide meaningful positive samples. This strategy comes in contrast with the current strategy where DA are the sole mean to incorporate known properties into the learned SSL representation. A key benefit of GPS-SSL lies in its applicability to any SSL method, e.g. SimCLR or BYOL. As a direct by-product, GPS-SSL also reduces the importance of DA to learn informative representations, a dependency that has been one of the major bottlenecks of SSL. We evaluate GPS-SSL along with multiple baseline SSL methods on multiple downstream datasets from different domains when the models use strong or minimal data augmentations. We show that when using strong DA, GPS-SSL outperforms the baselines on under- studied domains. Additionally, when using minimal augmentations –which is the most realistic scenario for which one does not know a priori the strong DA that aligns with the possible downstream tasks– GPS-SSL outperforms the baselines on all datasets by a significant margin. We believe that opening a new avenue to impact the SSL representations that is not solely based on altering the DA will open the door to multiple interesting research directions, greatly increasing the reach of SSL.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4387
Loading