Downstream Augmentation Generation For Contrastive Learning

Tomohiro Hayase, Suguru Yasutomi, Nakamasa Inoue

Abstract: Contrastive learning has become one of the most promising approaches for learning image representations. However, it heavily relies on heuristic data augmentation techniques, such as Gaussian blurring and color jittering, for making image pairs to be contrastively compared. These augmentations are not always appropriate for downstream tasks that each have their own camera and illumination settings. In this paper, we aim at improving the augmentation process and propose an augmentation generator, a network that learns to augment images for contrastive learning. Under the assumption that each downstream task has an optimal implicit augmentation function, the augmentation generator enhances the contrastive learning by estimating it. We demonstrate the effectiveness of our learning framework on two combined datasets, EMNIST-Omniglot and ImageNet-DAISO.