Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certification

Tong Chen, Jean B. Lasserre, Victor Magron, Edouard Pauwels

Keywords: Semialgebraic optimization, deep equilibrium model, robustness certification, Lipschitz constant

TL;DR: We approximate implicit ReLU monotone operator equilibrium networks using SDPs for certification purposes.

Abstract: Deep equilibrium models are based on implicitly defined functional relations and have shown competitive performance compared with the traditional deep networks. Monotone operator equilibrium networks (monDEQ) retain interesting performance with additional theoretical guaranties. Existing certification tools for classical deep networks cannot directly be applied to monDEQs for which much fewer tools exist. We introduce a semialgebraic representation for ReLU based monDEQs which allow to approximate the corresponding input output relation by semidefinite programs (SDP). We present several applications to network certification and obtain SDP models for the following problems : robustness certification, Lipschitz constant estimation, ellipsoidal uncertainty propagation. We use these models to certify robustness of monDEQs with respect to a general $L_p$ norm. Experimental results show that the proposed models outperform existing approaches for monDEQ certification. Furthermore, our investigations suggest that monDEQs are much more robust to $L_2$ perturbations than $L_{\infty}$ perturbations.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://github.com/NeurIPS2021Paper4075/SemiMonDEQ