Measuring and Improving Robustness of Deep Neural Networks
Keywords: robustness, generalization, out-of-distribution, adversarial
TL;DR: Measuring the robustness of Deep-Neural-Networks requires considering merely more than one perspective.
Abstract: Deep neural networks perform well on train data, but are often unable to adapt to
data distribution shifts. These are data which are rarely encountered, and thus are
under-represented in our training data. Examples of this includes data under ad-
verse weather conditions, and data which have been augmented with adversarial
perturbations. Estimating the robustness of models to data distribution shifts is im-
portant in enabling us to deploy them into safety critical applications with greater
assurance. Thus, we desire a measure which can be used to estimate robustness.
We define robustness in 4 ways: Generalization Gap, Test Accuracy (Clean &
Corrupted), and Attack Success Rate. A measure is said to be representative of
robustness when consistent (non-contradicting) relationships are found across all
4 robustness definitions. Through our empirical studies, we show that it is difficult
to measure robustness comprehensively across all definitions of robustness, as the
measure often behave inconsistently. While they can capture one aspect of robust-
ness, they often fail to do so in another aspect. Thus, we recommend that different
measures be used for different robustness definitions. Besides this, we also fur-
ther investigate the link between sharpness and robustness. We found that while
sharpness has some impact on robustness, this relationship is largely affected by
the choice of hyperparameters such as batch size.
Primary Area: other topics in machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1082
Loading