Adversarial Training Generalizes Data-dependent Spectral Norm Regularization

Anonymous

Sep 25, 2019 ICLR 2020 Conference Blind Submission readers: everyone Show Bibtex
  • TL;DR: We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks.
  • Abstract: We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we present a data-dependent variant of spectral norm regularization and prove that it is equivalent to adversarial training based on a specific $\ell_2$-norm constrained projected gradient ascent attack. This fundamental connection confirms the long-standing argument that a network's sensitivity to adversarial examples is tied to its spectral properties and hints at novel ways to robustify and defend against adversarial attacks. We provide extensive empirical evidence to support our theoretical results.
  • Keywords: Adversarial Robustness, Adversarial Training, Spectral Norm Regularization
0 Replies

Loading