Contrastive Learning Using Synthetic Images Generated from Real Images

Tenta Sasaya, Shintaro Yamamoto, Takashi Ida, Takahiro Takimoto

Abstract: The effectiveness of pre-training using large-scale natural image datasets has been demonstrated for situations in which there are limited available real images. However, some research has shown that models pre-trained using natural images cannot achieve sufficient performance on non-natural images taken under special circumstances or with special measurement devices. Although more general pre-training methods that use synthetic images such as random pattern images or noise images are a promising approach for such cases, their effectiveness depends on downstream tasks. To deal with this problem, we propose a contrastive learning framework using synthetic images generated from real images of downstream tasks to directly learn feature representations suitable for downstream tasks of non-natural images. Image classification experiments are performed on five non-natural image datasets mimicking real-world application with little available data, and these demonstrate that the proposed method achieves higher average classification accuracy compared with pre-training using ImageNet-1k or existing synthetic images with an improvement of over 6.5 points.