Augmenting cross-entropy with margin loss and applying moving average logits regularization to enhance adversarial robustness
Abstract: Despite significant progress in enhancing adversarial robustness, achieving a satisfactory level remains elusive, with a notable gap persisting between natural and adversarial accuracy. Recent studies have focused on mitigating inherent vulnerabilities in deep neural
networks (DNNs) by augmenting existing methodologies with additional data or reweighting strategies. However, most reweighting strategies often perform poorly against stronger attacks, and generating additional data often entails increased computational demands. Our work proposes an enhancement strategy that complements the cross-entropy loss with a margin-based loss for generating adversarial samples used in training and in the training loss function of promising methodologies. We suggest regularizing the training process by minimizing the discrepancy between the Exponential Moving Average (EMA) of adversarial and natural logits. Additionally, we introduce a novel training objective called Logits Moving Average Adversarial Training (LMA-AT). Our experimental results demonstrate the efficacy of our proposed method, which achieves a more favorable balance between natural and adversarial accuracy, thereby reducing the disparity between the two.
Submission Length: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=ZRybMTg9aB
Changes Since Last Submission: The significant changes made in response to the feedback on the previously rejected paper are summarized as follows:
1. **Revised Training Settings**:
- The parameter search range for α has been adjusted to address concerns raised by one of the reviewers. Previously, the range was set as {1.345, 2.345, 3.345, ..., 9.345}, which was perceived as unconventional. We have revised this to a more standard and formal range of integers [1, 10], specifically {1, 2, 3, ..., 10}.
- This modification required a reevaluation of most tables in the manuscript, with the exception of Table 15.
2. **Revised Results (Tables 3 and 4)**:
- Tables 3 and 4 have been updated to incorporate reviewer feedback. Additionally, we included further evaluations to address specific concerns, ensuring a more comprehensive presentation of results.
3. **New Table Added (Table 17)**:
- A new table (Table 17) has been introduced to provide a comparative analysis of our proposed method against TRADES and MART across various perturbation sizes. This addition offers a broader perspective on the robustness of our approach.
4. **Restructured Appendix Section**:
- The appendix has been reorganized to improve clarity and address structural concerns noted during the review.
These changes collectively aim to enhance the rigor and clarity of the manuscript while directly addressing reviewer concerns.
Assigned Action Editor: ~Yunhe_Wang1
Submission Number: 3948
Loading