Beyond Levels and Continuity: A New Statistical Method for DNNs Robustness Evaluation

Yanghao Zhang, Fu Wang, Tianle Zhang, Ronghui Mu, Xiaowei Huang, Wenjie Ruan

Keywords: Robustness; Point Process; Adversarial examples

TL;DR: An innovative statistical framework for assessing the robustness of DNNs, applicable to both vision and language models.

Abstract: Evaluating the robustness of deep neural networks (DNNs) is crucial in safety-critical areas, driving research into methods that accurately measure and enhance their resilience against adversarial attacks, specifically from a statistical perspective due to scalability issues faced by deterministic methods. Existing approaches based on independent sampling usually fail to directly capture such instances due to their rarity. Hence in this work, we treat the existence of adversarial examples as a rare event, and propose an innovative statistical framework for assessing the adversarial robustness of DNNs, called REPP. Our approach redefines the problem of calculating the occurrence of adversarial examples as the exponential of the mixture of a Poisson random variable and some potential geometric random variables. We adopt the point process to develop a Minimum Variance Unbiased Estimator (MVUE) to accurately estimate the likelihood of encountering adversarial examples, with an upper bound of the true probability with high confidence. Unlike existing rare-event methods based on Multi-level Splitting, REPP does not require the inherent level concept or the continuity condition of the cumulative distribution function (CDF) within DNNs. This adaptation allows for practical application across both computer vision and natural language processing tasks. Experimental results demonstrate that our method is more flexible and effective, offering a more reliable robustness evaluation than existing statistical approaches.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3960