OpenMixup: Open Mixup Toolbox and Benchmark for Visual Representation Learning

Siyuan Li, Zedong Wang, Zicheng Liu, Di Wu, Cheng Tan, Weiyang Jin, Stan Z. Li

Keywords: Data Augmentation, Image Classification, Mixup, Vision Transformer, Mixup

TL;DR: This paper presents the first comprehensive mixup benchmarking study, OpenMixup, for supervised visual classification, which offers a unified mixup-based model design and training codebase.

Abstract: Mixup augmentation has emerged as a powerful technique for improving the generalization ability of deep neural networks. However, the lack of standardized implementations and benchmarks has hindered progress, resulting in poor reproducibility, unfair comparisons, and conflicting insights. In this paper, we introduce OpenMixup, the \textit{first} mixup augmentation benchmark for visual representation learning, where 18 representative mixup baselines are trained \textit{from scratch} and systematically evaluated on 11 image datasets across varying scales and granularity, spanning fine-grained scenarios to complex non-iconic scenes. We also open-source a modular codebase for streamlined mixup method design, training, and evaluations, which comprises a collection of widely-used vision backbones, optimization policies, and analysis toolkits. Notably, the codebase not only underpins all our benchmarking but supports broader mixup applications beyond classification, such as self-supervised learning and regression tasks. Through extensive experiments, we present insights on performance-complexity trade-offs and identify preferred mixup strategies for different needs. To the best of our knowledge, OpenMixup has contributed to a number of studies in the mixup community. We hope this work can further advance reproducible mixup research and fair comparisons, thereby laying a solid foundation for future progress. The source code is publicly available at \url{https://github.com/Westlake-AI/openmixup}.

Submission Number: 630