Gradient semi-masking for improving adversarial robustness

Xinlei Liu, Tao Hu, Peng Yi, Baolin Li, Jichao Xie, Hailong Ma

Abstract: In gradient masking, certain complex signal processing and probabilistic optimization strategies exhibit favorable characteristics such as nonlinearity, irreversibility, and feature preservation, thereby providing new solutions for adversarial defense. Inspired by this, this paper proposes a plug-and-play gradient semi-masking module (GSeM) to improve the adversarial robustness of neural networks. GSeM primarily contains a feature straight-through pathway that allows for normal gradient propagation and a feature mapping pathway that interrupts gradient flow. The multi-pathway and semi-masking characteristics cause GSeM to exhibit opposing behaviors when processing data and gradients. Specifically, during data processing, GSeM compresses the state space of features while introducing white noise augmentation. However, during gradient processing, it leads to inefficient updates to certain parameters and ineffective generation of training examples. To address this shortcoming, we correct gradient propagation and introduce gradient-corrected adversarial training. Extensive experiments demonstrate that GSeM differs fundamentally from earlier gradient masking methods: it can genuinely enhance the adversarial defense performance of neural networks, surpassing previous state-of-the-art approaches.