Sharpness-Aware Minimization Alone can Improve Adversarial Robustness

Zeming Wei, Jingyu Zhu, Yihao Zhang

Keywords: Adversarial Robustness, Sharpness-Aware Minimization

TL;DR: We show that using Sharpness-Aware Minimization (SAM) alone can improve adversarial robustness, which is a promising lightweight substitute for Adversarial Training (AT) under certain requirements.

Abstract: Sharpness-Aware Minimization (SAM) is an effective method for improving generalization ability by regularizing loss sharpness. In this paper, we explore SAM in the context of adversarial robustness. We find that using only SAM can achieve superior adversarial robustness without sacrificing clean accuracy compared to standard training, which is an unexpected benefit. We also discuss the relation between SAM and adversarial training (AT), a popular method for improving the adversarial robustness of DNNs. In particular, we show that SAM and AT differ in terms of perturbation strength, leading to different accuracy and robustness trade-offs. We provide theoretical evidence for these claims in a simplified model. Finally, while AT suffers from decreased clean accuracy and computational overhead, we suggest that SAM can be regarded as a lightweight substitute for AT under certain requirements. Code is available at https://github.com/weizeming/SAM_AT.

Submission Number: 7