A dynamical system perspective for lipschitz neural networks

Blaise Delattre, Laurent Meunier, Alexandre Araujo, Alexandre Allauzen

Abstract:

The Lipschitz constant of neural networks has
been established as a key quantity to enforce the
robustness to adversarial examples. In this paper,
we tackle the problem of building 1-Lipschitz
Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method
to build 1-Lipschitz Neural Networks and show
that some previous approaches are special cases of
this framework. Then, we extend this reasoning
and show that ResNet flows derived from convex potentials define 1-Lipschitz transformations,
that lead us to define the Convex Potential Layer
(CPL). A comprehensive set of experiments on
several datasets demonstrates the scalability of
our architecture and the benefits as an ℓ2-provable
defense against adversarial examples. Our
code is available at https://github.com/
MILES-PSL/Convex-Potential-Layer