The shape and simplicity biases of adversarially robust ImageNet-trained CNNs

Peijie Chen, Chirag Agarwal, Anh Nguyen

Keywords: shape bias, texture bias, interpretability, smoothness, visualization

Abstract:

Adversarial training has been the topic of dozens of studies and a leading method for defending against adversarial attacks.
Yet, it remains largely unknown (a) how adversarially-robust ImageNet classifiers (R classifiers) generalize to out-of-distribution examples; and (b) how their generalization capability relates to their hidden representations. In this paper, we perform a thorough, systematic study to answer these two questions across AlexNet, GoogLeNet, and ResNet-50 architectures. We found that while standard ImageNet classifiers have a strong texture bias, their R counterparts rely heavily on shapes. Remarkably, adversarial training induces three simplicity biases into hidden neurons in the process of ``"robustifying" networks. That is, each convolutional neuron in R networks often changes to detecting (1) pixel-wise smoother patterns i.e. a mechanism that blocks high-frequency noise from passing through the network; (2) more lower-level features i.e. textures and colors (instead of objects); and (3) fewer types of inputs. Our findings reveal the interesting mechanisms that made networks more adversarially robust and also explain some recent findings e.g. why R networks benefit from much larger capacity and can act as a strong image prior in image synthesis.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: adversarial training makes networks more robust by smoothing out kernels and reducing the space of active inputs of each neuron

Reviewed Version (pdf): https://openreview.net/references/pdf?id=3C6YaIBFpn