Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup

Siyuan Li, Zicheng Liu, Zedong Wang, Di Wu, Zihan Liu, Yang Liu, Baigui Sun, Stan Z. Li

Keywords: Data Augmentation, Image Classification, Self-supervised Learning, Mixup, Representation Learning

TL;DR: We propose a learnable scenario-agnostic mixup (SAMix) methods for self-supervised and various supervised vision representation learning.

Abstract:

Mixup is a hit data-dependent augmentation technique that entails two sub-tasks: mixed sample generation and classification. This paper comprehensively studies the objective of mixup generation and proposes \textbf{S}cenario-\textbf{A}gnostic \textbf{Mix}up (SAMix) to address the two remaining challenges in this field at once:
\textbf{(i) Huge performance variation over scenarios caused by trivial solutions.} The objective of mixup generation narrows to selected sample pairs rather than the whole observed data manifold, which gives rise to the hassle of trivial solutions, resulting in drastic variations in sample mixing performance over different scenarios.
\textbf{(ii) Self-supervised learning (SSL) dilemma for online training policies.} While recent online training policies can generate out-of-manifold samples on supervised learning (SL), simply applying them to SSL scenarios leads to subpar performance.
We hypothesize and verify the objective function of mixup generation as optimizing \textit{local smoothness} between two mixed classes subject to \textit{global discrimination} from the other classes. Thus, we propose $\eta$-balanced mixup loss for complementary learning of the two sub-objectives. For the generation model, a label-free generator, Mixer, is designed to generate non-trivial mixed samples with great transferability. To reduce the computational cost from online training, we further introduce a pre-trained version, SAMix$^\mathcal{P}$, which is more applicable and achieves more favorable generalizability. Extensive experiments on 12 SL and SSL image benchmarks show the consistent superiority of SAMix compared with state-of-the-art methods.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 147