Adversarial Fine-tuning of Compressed Neural Networks for Joint Improvement of Robustness and Efficiency

TMLR Paper5586 Authors

Abstract: As deep learning (DL) models are increasingly being integrated into our everyday lives, ensuring their safety by making them robust against adversarial attacks has become increasingly critical. DL models have been found to be susceptible to adversarial attacks by introducing small, targeted perturbations to disrupt the input data. Adversarial training has been presented as a mitigation strategy that can result in more robust models. This adversarial robustness comes with additional computational costs required to design adversarial attacks during training. The two objectives -- adversarial robustness and computational efficiency -- then appear to be in conflict with each other. In this work, we explore the effects of neural network compression on adversarial robustness. We specifically explore the effects of fine-tuning on compressed models, and present the trade-off between standard fine-tuning and adversarial fine-tuning. Our results show that {\em adversarial fine-tuning} of compressed models can yield large improvements to their robustness performance. We present experiments on several benchmark datasets showing that adversarial fine-tuning of compressed models can achieve robustness performance comparable to adversarially trained models, while also improving computational efficiency.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=PJQ4b2zvvF

Changes Since Last Submission:

We made some major and minor changes based on the reviewers' feedbacks from our previous TMLR submission.

1. Add comprehensive experiments for larger datasets and models.
2. Use AutoAttack to evaluate adversarial robustness.
3. Include [1] in the related works, and clarify the relation with our contribution.

[1] Lin, Ji, Chuang Gan, and Song Han. "Defensive quantization: When efficiency meets robustness." arXiv preprint arXiv:1904.08444 (2019).

Assigned Action Editor: ~Amir-massoud_Farahmand1

Submission Number: 5586