Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

Xiao Yang, Yinpeng Dong, Wenzhao Xiang, Tianyu Pang, Hang Su, Jun Zhu

Keywords: Adversarial attacks, robust evaluation

Abstract: The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness. However, the progress is usually hampered by insufficient robustness evaluations. As the de facto standard to evaluate adversarial robustness, adversarial attacks typically solve an optimization problem of crafting adversarial examples with an iterative process. But the existing attacks are usually limited by using hand-designed optimization algorithms, leading to less accurate robustness evaluations. In this paper, we propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the optimizer in adversarial attacks parameterized by a recurrent neural network, which is trained over a class of data samples and defense models to produce effective update directions during adversarial example generation. Furthermore, we develop a model-agnostic training algorithm to improve the generalization ability of the learned optimizer when attacking unseen defenses. Our approach can be flexibly incorporated with various attacks and consistently improves their performance. Extensive experiments demonstrate the effectiveness and efficiency of the learned attacks by MAMA, e.g., MAMA achieves x2 speedup over the state-of-the-art AutoAttack while obtaining lower robust test accuracy on all adopted defense models. Therefore, MAMA leads to a more reliable and efficient evaluation of adversarial robustness.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Social Aspects of Machine Learning (eg, AI safety, fairness, privacy, interpretability, human-AI interaction, ethics)

Community Implementations: [ 11 code implementations](https://www.catalyzex.com/paper/model-agnostic-meta-attack-towards-reliable/code)