How Do Augmentations with Label Smoothing Enhance Model Robustness?

Weebum Yoo, Sung Whan Yoon

Keywords: Learning theory, Model robustness, Data augmentation, label smoothing, Generalization

Abstract: Model robustness indicates a model's capability to generalize well on unforeseen distributional shifts, including data corruption, adversarial attacks, and domain shifts. One of the most prevalent and effective ways to enhance the robustness often involves data augmentations and label smoothing techniques. Despite the great success of the related approaches in diverse practices, a unified theoretical understanding of their efficacy in improving model robustness is lacking. We offer a theoretical framework to clarify how augmentations, label smoothing, or their combination enhance model robustness through the lens of loss surface flatness, generalization bound, and adversarial robustness. Specifically, we first formally bridge the diversified data distribution via augmentations to the flatter minima on the parameter space, which directly links to the improved generalization capability. Moreover, we further bridge augmentations with label smoothing, which softens the confidence of the target label, to the improved adversarial robustness. We broadly confirm our theories through extensive simulations on the existing common corruption and adversarial robustness benchmarks based on the CIFAR and tinyImageNet datasets, as well as various domain generalization benchmarks.

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2320