Adversarial Training Can Hurt Generalization

Aditi Raghunathan*, Sang Michael Xie*, Fanny Yang, John Duchi, Percy Liang

Keywords: robustness, generalization, tradeoff

Abstract: While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tradeoff between standard and robust accuracy, but only in the setting where no predictor performs well on both objectives in the infinite data limit. In this paper, we show that even when the optimal predictor with infinite data performs well on both objectives, a tradeoff can still manifest itself with finite data. Furthermore, since our construction is based on a convex learning problem, we rule out optimization concerns, thus laying bare a fundamental tension between robustness and generalization. Finally, we show that robust self-training mostly eliminates this tradeoff by leveraging unlabeled data.

TL;DR: Even if there is no tradeoff in the infinite data limit, adversarial training can have worse standard accuracy even in a convex problem.