Certified Robustness for Deep Equilibrium Models via Interval Bound Propagation

Colin Wei, J Zico Kolter

Keywords: deep equilibrium models, certified robustness, interval bound propagation

Abstract: Deep equilibrium layers (DEQs) have demonstrated promising performance and are competitive with standard explicit models on many benchmarks. However, little is known about certifying robustness for these models. Inspired by interval bound propagation (IBP), we propose the IBP-MonDEQ layer, a DEQ layer whose robustness can be verified by computing upper and lower interval bounds on the output. Our key insights are that these interval bounds can be obtained as the fixed-point solution to an IBP-inspired equilibrium equation, and furthermore, that this solution always exists and is unique when the layer obeys a certain parameterization. This fixed point can be interpreted as the result of applying IBP to an infinitely deep, weight-tied neural network, which may be of independent interest, as IBP bounds are typically unstable for deeper networks. Our empirical comparison reveals that models with IBP-MonDEQ layers can achieve comparable $\ell_{\infty}$ certified robustness to similarly-sized fully explicit networks.

One-sentence Summary: To develop certifiably robust deep equilibrium (DEQ) models, we propose the IBP-MonDEQ layer, a DEQ layer where interval bounds on the output can be obtained by solving an additional fixed-point equation inspired by interval bound propagation.