Robustifying $\ell_\infty$ Adversarial Training to the Union of Perturbation Models

Ameya Patil, Michael Tuttle, Alex Schwing, Naresh Shanbhag

Keywords: adversarial examples, adversarial robustness, efficient adversarial training

Abstract: Classical adversarial training (AT) frameworks are designed to achieve high adversarial accuracy against a single attack type, typically $\ell_\infty$ norm-bounded perturbations. Recent extensions in AT have focused on defending against the union of multiple perturbation models but this benefit is obtained at the expense of a significant (up to $10\times$) increase in training complexity over single-attack $\ell_\infty$ AT. In this work, we expand the capabilities of widely popular $\ell_\infty$ single-attack AT frameworks to providing robustness to the union of ($\ell_\infty, \ell_2, \ell_1$) perturbations while preserving their training efficiency. Our technique, referred to as Shaped Noise Augmented Processing (SNAP), exploits a well-established byproduct of single-attack AT frameworks -- the reduction in the curvature of the decision boundary of networks. SNAP prepends a given deep net with a shaped noise augmentation layer whose distribution is learned along with network parameters using any standard single-attack AT. As a result, SNAP enhances adversarial accuracy of ResNet-18 on CIFAR-10 against the union of ($\ell_\infty, \ell_2, \ell_1$) perturbation models by 14%-to-20% for four state-of-the-art (SOTA) single-attack $\ell_\infty$ AT frameworks, and, for the first time, establishes a benchmark for ResNet-50 and ResNet-101 on ImageNet.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

TL;DR: In this work, we expand the capabilities of widely popular and efficient $\ell_\infty$ adversarial training frameworks to providing robustness to the union of ($\ell_\infty, \ell_2, \ell_1$) perturbations while preserving their training efficiency.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/robustifying-ell-infty-adversarial-training/code)