Robust Design of Deep Neural Networks Against Adversarial Attacks Based on Lyapunov Theory

Arash Rahnama, André T. Nguyen, Edward Raff

Abstract: Deep neural networks (DNNs) are vulnerable to subtle adversarial perturbations applied to the input. These adversarial perturbations, though imperceptible, can easily mislead the DNN. In this work, we take a control theoretic approach to the problem of robustness in DNNs. We treat each individual layer of the DNN as a nonlinear system and use Lyapunov theory to prove stability and robustness locally. We then proceed to prove stability and robustness globally for the entire DNN. We develop empirically tight bounds on the response of the output layer, or any hidden layer, to adversarial perturbations added to the input, or to any preceding hidden layer. We show how the spectral norm of the weight matrix for an individual layer relates to Lyapunov properties of that layer, and consequently to the local and global stability and robustness of the DNN. Our results give new insights into how spectral norm regularization can mitigate the adversarial effects. Finally, we evaluate the power of our approach on a variety of data sets and network architectures and against some of the well-known adversarial attacks.