DAMO: Dynamic Adversarial defense with Multi-branch Orthogonality
Keywords: Adversarial Robustness, Dynamic Neural Network
Abstract: Deep neural networks (DNNs) are vulnerable to adversarial examples, in which DNNs are misled to false outputs due to inputs containing imperceptible perturbations. Ensemble adversarial training, which integrates multiple robust classifiers where each classifier is trained by an adversarial training strategy, becomes one of the widely used strategies to increase the model's robustness. However, we observe that ensemble methodologies exhibit certain limitations whereby erroneous predictions can arise once more than half of the employed classifiers are breached. This suggests that ensemble methods alone may not be sufficient for achieving perfect classification accuracy. Therefore, we raise an intriguing question: can perfect classification be achieved with only one accurate classifier remaining?
In this paper, we propose Dynamic Adversarial defense with Multi-branch Orthogonality (DAMO). Different from the traditional ensemble adversarial defense, which calculates the prediction confidence for each classifier in the model and votes to return the final prediction, we design a dynamic routing module to dynamically select a robust classifier from the model members for inference of each adversarial sample. The proposed method shows an increased ability to correctly predict the output value, even if only one classifier is not successfully attacked. Then, considering the classifiers in the ensemble model may have large similarities and cause an adversarial sample to trigger all classifiers into error, we propose Branch Orthogonality~(BO) loss, which is used to increase the diversity of each classifier in the ensemble model. Exhaustive experiments are conducted to show that our method goes beyond all state-of-the-art methods. Compared to SOTA baselines, our models achieve 66.27\% and 40.12\% robust accuracy on CIFAR-10 and CIFAR-100 (improving upon the state-of-the-art by +6.20\% and +7.69\%).
Primary Area: general machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7479
Loading