Enhancing fast adversarial training with momentum-driven initialization and max-norm regularization for robust deep learning models

Anjum Iqbal, Weiqiang Kong, Yasir Iqbal, Umer Sadiq Khan, Shah Fahad Khan, Ijaz Hussain

Abstract: Fast adversarial training (FAT) has emerged as an effective strategy to enhance the robustness of deep learning models against adversarial attacks. However, catastrophic overfitting remains a significant challenge, where models experience a sudden deterioration in their adversarial robustness. To address this issue, we propose a novel approach combining momentum-driven initialization and max-norm regularization, termed MIMR. Our momentum-driven initialization method maintains high-quality adversarial examples throughout training by leveraging previous perturbations, without increasing computational costs. Furthermore, max-norm regularization restricts gradient updates, fostering a more stable and generalizable adversarial training framework. Theoretical evaluations support our proposed methods, which improve loss function smoothness. Experiments on benchmark datasets, including CIFAR-10, CIFAR-100, and Tiny ImageNet, demonstrate that MIMR not only mitigates catastrophic overfitting but also achieves higher classification accuracy under various attacks, such as PGD, C&W, and AutoAttack. Specifically, MIMR performs exceptionally well under strong attacks, achieving 49.44% accuracy on CIFAR-10, 25.59% on CIFAR-100, and 17.26% on Tiny ImageNet for AutoAttack, showcasing superior robustness compared to existing methods. The source code is available at https://github.com/anjumiqbal06/MIMR.

External IDs:dblp:journals/vc/IqbalKIKKH25