Shape-Texture Debiased Neural Network Training

Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Yuille, cihang xie

Keywords: data augmentation, representation learning, debiased training

Abstract:

Shape and texture are two prominent and complementary cues for recognizing objects. Nonetheless, Convolutional Neural Networks are often biased towards either texture or shape, depending on the training dataset. Our ablation shows that such bias degenerates model performance. Motivated by this observation, we develop a simple algorithm for shape-texture debiased learning. To prevent models from exclusively attending on a single cue in representation learning, we augment training data with images with conflicting shape and texture information (eg, an image of chimpanzee shape but with lemon texture) and, most importantly, provide the corresponding supervisions from shape and texture simultaneously.

Experiments show that our method successfully improves model performance on several image recognition benchmarks and adversarial robustness. For example, by training on ImageNet, it helps ResNet-152 achieve substantial improvements on ImageNet (+1.2%), ImageNet-A (+5.2%), ImageNet-C (+8.3%) and Stylized-ImageNet (+11.1%), and on defending against FGSM adversarial attacker on ImageNet (+14.4%). Our method also claims to be compatible with other advanced data augmentation strategies, eg, Mixup, and CutMix. The code is available here: https://github.com/LiYingwei/ShapeTextureDebiasedTraining.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: Training CNNs to acquire a debiased shape-texture representation improves image recognition.

Code: [ LiYingwei/ShapeTextureDebiasedTraining](https://github.com/LiYingwei/ShapeTextureDebiasedTraining)

Data: [ImageNet](https://paperswithcode.com/dataset/imagenet), [ImageNet-A](https://paperswithcode.com/dataset/imagenet-a), [ImageNet-C](https://paperswithcode.com/dataset/imagenet-c), [ImageNet-Sketch](https://paperswithcode.com/dataset/imagenet-sketch), [Stylized ImageNet](https://paperswithcode.com/dataset/stylized-imagenet)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2010.05981/code)