Understanding ReLU Network Robustness Through Test Set Certification Performance

Nicola Franco, Jeanette Miriam Lorenz, Karsten Roscher, Stephan Günnemann

Keywords: Robustness Certificates, Robust Machine Learning, Out-Of-Distribution Detection

Abstract:

Neural networks might exhibit weak robustness against input perturbations within the learning distribution and become more severe for distributional shifts or data outside the distribution.
For their safer use, robustness certificates provide formal guarantees to the stability of the prediction in the vicinity of the input.
However, the relationship between correctness and robustness remains unclear.
In this work, we investigate the unexpected outcomes of verification methods applied to piecewise linear classifiers for clean, perturbed, in- and out-of-distribution samples.
In our experiments, we conduct a thorough analysis for image classification tasks and show that robustness certificates are strongly correlated with prediction correctness for in-distribution data.
In addition, we provide a theoretical demonstration that formal verification methods robustly certify samples sufficiently far from the training distribution.
These results are integrated with an experimental analysis and demonstrate their weakness compared to standard out-of-distribution detection methods.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning

TL;DR: Robustness certificates for ReLU networks are strongly correlated with network accuracy for data in-distribution and are highly unreliable for data out-of-distribution.