Improved Gradient based Adversarial Attacks for Quantized Networks

Kartik Gupta, Thalaiyasingam Ajanthan

Keywords: binary neural network, gradient masking, fake robustness, temperature scaling, adversarial attack, signal propagation

Abstract: Neural network quantization has become increasingly popular due to efficient memory consumption and faster computation resulting from bitwise operations on the quantized networks. Even though they exhibit excellent generalization capabilities, their robustness properties are not well-understood. In this work, we systematically study the robustness of quantized networks against gradient based adversarial attacks and demonstrate that these quantized models suffer from gradient vanishing issues and show a fake sense of robustness. By attributing gradient vanishing to poor forward-backward signal propagation in the trained network, we introduce a simple temperature scaling approach to mitigate this issue while preserving the decision boundary. Despite being a simple modification to existing gradient based adversarial attacks, experiments on CIFAR-10/100 datasets with multiple network architectures demonstrate that our temperature scaled attacks obtain near-perfect success rate on quantized networks while outperforming original attacks on adversarially trained models as well as floating point networks.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: In this work, we systematically study the robustness of quantized networks against gradient based adversarial attacks and propose PGD++ attack variants to overcome existing gradient masking issues in BNNs.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2003.13511/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=HlEOskZeSJ