Empirical Analysis of Upper Bounds of Robustness Distributions using Adversarial Attacks

Aaron Berger, Nils Eberhardt, Annelot Willemijn Bosman, Henning Duwe, Jan N. van Rijn, Holger Hoos

Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.

Tracks: Main Track

Keywords: Neural Network Verification, Adversarial Robustness, Distribution Analysis, Adversarial Examples, Neural Network Verification

TL;DR: We investigate the ability of adversarial attack methods to provide a tight upper bound for robustness distributions generated with expensive complete verification.

Abstract:

This study examines the effectiveness of adversarial attacks in determining upper bounds for robustness distributions for neural networks.
While complete neural network verification techniques can provide exact safety margins, their computational cost limits scalability.
To address this, we evaluate multiple adversarial attack methods, including FGSM, PGD, AutoAttack and FAB, comparing them to a state-of-the-art verification technique, $\alpha, \beta$-CROWN.
Using the MNIST dataset, we demonstrate that adversarial attacks yield computationally efficient and tight upper bounds for robustness distributions.
We assess the trade-offs between running time, accuracy and the quality of the bounds obtained through our approach.
The results highlight complementarities between verification and attack methods: Attacks achieve near-optimal upper bounds at a significantly reduced computational cost.
These findings open opportunities for large-scale robustness analysis while acknowledging limitations in safety guarantees inherent to the approximation techniques on which our approach is based.

Submission Number: 20