Adversarial Visual Robustness by Causal Intervention

Kaihua Tang, Mingyuan Tao, Hanwang Zhang

Keywords: Adversarial Robustness, Causality, Instrumental Variable

Abstract: Adversarial training is the de facto most promising defense against adversarial examples. Yet, its passive nature inevitably prevents it from being immune to unknown attackers. To achieve a proactive defense, we need a more fundamental understanding of adversarial examples, beyond the popular bounded threat model. In this paper, we provide a causal viewpoint of adversarial vulnerability: the cause is the spurious correlations ubiquitously existing in learning, i.e., the confounding effect, where attackers are precisely exploiting these effects. Therefore, a fundamental solution for adversarial robustness is by causal intervention. As visual confounders are imperceptible in general, we propose to use the instrumental variable that achieves causal intervention without the need for confounder observation. We term our robust training method as Causal intervention by instrumental Variable (CiiV). It's a causal regularization that 1) augments the image with multiple retinotopic centers and 2) encourages the model to learn causal features rather than local confounding patterns by favoring features linearly responding to spatial interpolations. Extensive experiments on a wide spectrum of attackers and settings applied in CIFAR-10, CIFAR-100, and mini-ImageNet demonstrate that CiiV is robust to adaptive attacks, including the recent Auto-Attack. Besides, as a general causal regularization, it can be easily plugged into other methods to further boost their robustness. Codes are available in supplementary materials.

One-sentence Summary: This paper proposes a plug-and-play causal regularization method to improve the adversarial robustness, which is inspired by the causal intervention using instrumental variable.

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/arxiv:2106.09534/code)