Learning Contextual Perturbation Budgets for Training Robust Neural Networks

Jing Xu, Zhouxing Shi, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, Liwei Wang

Keywords: adversarial robustness, certified robustness, certfied robust training

Abstract: Existing methods for training robust neural networks generally aim to make models uniformly robust on all input dimensions. However, different input dimensions are not uniformly important to the prediction. In this paper, we propose a novel framework to train certifiably robust models and learn non-uniform perturbation budgets on different input dimensions, in contrast to using the popular $\ell_\infty$ threat model. We incorporate a perturbation budget generator into the existing certified defense framework, and perform certified training with generated perturbation budgets. In comparison to the radius of $\ell_\infty$ ball in previous works, the robustness intensity is measured by robustness volume which is the multiplication of perturbation budgets on all input dimensions. We evaluate our method on MNIST and CIFAR-10 datasets and show that we can achieve lower clean and certified errors on relatively larger robustness volumes, compared to methods using uniform perturbation budgets. Further with two synthetic datasets constructed from MNIST and CIFAR-10, we also demonstrate that the perturbation budget generator can produce semantically-meaningful budgets, which implies that the generator can capture contextual information and the sensitivity of different features in input images.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: We propose a novel algorithm to learn contextual and non-uniform perturbation budgets and train certifiably robust neural networks with learned perturbation budgets.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=crtmzXu6RW