Boosting Unsupervised Contrastive Learning Using Diffusion-Based Data Augmentation From Scratch
Keywords: Data Augmentation; AI for Science; Unsupervised Contrastive Learning
Abstract: Unsupervised Contrastive learning has gained prominence in fields such as vision, natural language processing, and biology, leveraging predefined positive and negative samples for representation learning. Data augmentation, categorized into hand-designed and model-based methods, has been identified as a crucial component for enhancing contrastive learning. However, hand-designed methods require human expertise in domain-specific data while sometimes distorting the meaning of the data. In contrast, model-based methods, such as generative models, usually require supervision or large-scale external data. To address the problems presented above, this paper proposes DiffAug, a novel unsupervised contrastive learning technique with diffusion mode-based positive data generation. DiffAug consists of a semantic encoder and a conditional diffusion model; the conditional diffusion model generates new positive samples conditioned on the semantic encoding to serve the training of unsupervised contrast learning. With the help of iterative training of the semantic encoder and diffusion model, DiffAug improves the representation ability in an uninterrupted and unsupervised manner. Experimental evaluations show that DiffAug outperforms hand-designed augmentation and classical representation learning methods in classification and clustering tasks on visual and biological datasets, highlighting its potential for generalizing unsupervised learning techniques. The code for review is released at https://anonymous.4open.science/r/diffaug_review-804E.
Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 250
Loading