Fantastic Robustness Measures: The Secrets of Robust Generalization

Hoki Kim, Jinseong Park, Yujin Choi, Jaewook Lee

Keywords: Adversarial Robustness, Generalization, Measures

TL;DR: This study conducts a large-scale analysis over 1,300 models to investigate when and how existing measures (e.g., margin, smoothness, flatness) are effective in estimating robust generalization.

Abstract: Adversarial training has become the de-facto standard method for improving the robustness of models against adversarial examples. However, robust overfitting remains a significant challenge, leading to a large gap between the robustness on the training and test datasets. To understand and improve robust generalization, various measures have been developed, including margin, smoothness, and flatness-based measures. In this study, we present a large-scale analysis of robust generalization to empirically verify whether the relationship between these measures and robust generalization remains valid in diverse settings. We demonstrate when and how these measures effectively capture the robust generalization gap by comparing over 1,300 models trained on CIFAR-10 under the $L_\infty$ norm and further validate our findings through an evaluation of more than 100 models from RobustBench across CIFAR-10, CIFAR-100, and ImageNet. We hope this work can help the community better understand adversarial robustness and motivate the development of more robust defense methods against adversarial attacks.

Submission Number: 6931