Universal Pyramid Adversarial Training for Improved ViT Performance
Abstract: Recently, Pyramid Adversarial training has been shown to be very effective for improving clean accuracy and distribution-shift robustness of vision transformers. However, due to the iterative nature of adversarial training, the technique is up to 7 times more expensive than standard training. To make the method more efficient, we propose Universal Pyramid Adversarial training, where we learn a single pyramid adversarial pattern shared across the whole dataset instead of the sample-wise patterns. With our proposed technique, we decrease the computational cost of Pyramid Adversarial training by up to 70\% while retaining the majority of its benefit on clean performance and distribution-shift robustness. In addition, to the best of our knowledge, we are also the first to find that universal adversarial training can be leveraged to improve clean model performance.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: updated paper in response to reviewer's suggestions.
Assigned Action Editor: ~Charles_Xu1
Submission Number: 1042
Loading