On Frank-Wolfe Adversarial Training

Theodoros Tsiligkaridis, Jay Roberts

Keywords: adversarial training, robustness, loss landscape, Frank-Wolfe optimization

TL;DR: This paper introduces adaptive adversarial training with Frank-Wolfe optimization (FW-AT) that reveals a geometric connection between loss landscape and attack distortion, provides strong robustness at lower training times in comparison to PGD-AT.

Abstract: We develop a theoretical framework for adversarial training (AT) with FW optimization (FW-AT) that reveals a geometric connection between the loss landscape and the distortion of $\ell_\infty$ FW attacks (the attack's $\ell_2$ norm). Specifically, we show that high distortion of FW attacks is equivalent to low variation along the attack path. It is then experimentally demonstrated on various deep neural network architectures that $\ell_\infty$ attacks against robust models achieve near maximal $\ell_2$ distortion. To demonstrate the utility of our theoretical framework we develop FW-Adapt, a novel adversarial training algorithm which uses simple distortion measure to adapt the number of attack steps during training. FW-Adapt provides strong robustness against white- and black-box attacks at lower training times than PGD-AT.