Fast Reliability Estimation for Neural Networks with Adversarial Attack-Driven Importance Sampling

Karim TIT, Teddy Furon

Keywords: Deep Neural Networks, Reliability, Adversarial Attacks, Importance Sampling

TL;DR: Combining adversarial attacks and importance sampling for fast evaluation of statistical robustness

Abstract: This paper introduces a novel approach to evaluate the reliability of Neural Networks (NNs) by integrating adversarial attacks with Importance Sampling (IS), enhancing the assessment's precision and efficiency. Leveraging adversarial attacks to guide IS, our method efficiently identifies vulnerable input regions, offering a more directed alternative to traditional Monte Carlo methods. While comparing our approach with classical reliability techniques like FORM and SORM, and with classical rare event simulation methods such as Cross-Entropy IS, we acknowledge its reliance on the effectiveness of adversarial attacks and its inability to handle very high-dimensional data such as ImageNet. Despite these challenges, our comprehensive empirical validations on the datasets the MNIST and CIFAR10 demonstrate the method's capability to accurately estimate NN reliability for a variety of models. Our research not only presents an innovative strategy for reliability assessment in NNs but also sets the stage for further work exploiting the connection between adversarial robustness and the field of statistical reliability engineering.

List Of Authors: Tit, Karim and Furon, Teddy

Code Url: https://github.com/karimtito/stat_reliability_measure/tree/exp_thesis

Submission Number: 43