DeepSplit: Scalable Verification of Deep Neural Networks via Operator Splitting

Shaoru Chen, Eric Wong, J Zico Kolter, Mahyar Fazlyab

Keywords: neural network verification, operator splitting, ADMM

Abstract: Analyzing the worst-case performance of deep neural networks against input perturbations amounts to solving a large-scale non-convex optimization problem, for which several past works have proposed convex relaxations as a promising alternative. However, even for reasonably-sized neural networks, these relaxations are not tractable, and so must be replaced by even weaker relaxations in practice. In this work, we propose a novel operator splitting method that can directly solve a convex relaxation of the problem to high accuracy, by splitting it into smaller sub-problems that often have analytical solutions. The method is modular, scales to very large problem instances, and compromises of operations that are amenable to fast parallelization with GPU acceleration. We demonstrate our method in obtaining tighter bounds on the worst-case performance of large convolutional networks in image classification and reinforcement learning settings.

One-sentence Summary: Verify robustness of deep neural networks via operator splitting to promote scalability