Advancing the Adversarial Robustness of Neural Networks from the Data Perspective

Monty Maximilian Zühlke, Daniel Kudenko

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: supervised representation learning, representation learning for computer vision, visualization or interpretation of learned representations

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We extract information from data, relate it to learned representations and use it to boost the adversarial robustness of a model.

Abstract: Robustness in machine learning is a widespread concept and one of the pillars of trustworthiness, ranging from a model's resistance to noise---benign and adversarial---to the reliability of benchmarking. In this work, we analyse the robustness of labelled data which we argue corresponds to the data manifold's curvature as perceived by a model during training and thus establish a connection to its adversarial robustness. This view provides an intuitive explanation for our empirical results showing that neural networks acquire adversarial robustness much slower in the least robust regions. In combination with minor adjustments to the learning rate, the new concept offers a means to emphasise these regions during training and increase the model's overall adversarial robustness, even when using identical computational resources.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3684