Improving Branching in Neural Network Verification with Bound Implication Graph

Zhuolin Yang, Kaidi Xu, Bo Li, Huan Zhang

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: neural network verification, adversarial robustness, branch and bound

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We improve branch and bound based verification by exploiting the dependencies among neurons and building an implication graph.

Abstract:

Many state-of-the-art neural network verifiers for ReLU networks rely on Branch and Bound (BaB)-based methods. They branch ReLUs into positive (active) and negative (inactive) parts, and bound each subproblem independently. Since the cost of verification heavily depends on the number of subproblems, reducing the total number of branches is the key to verifying neural networks efficiently. In this paper, we consider \emph{bound implications} during branching - i.e., when one or more ReLU neurons are branched into the active (or inactive) case, they may imply that a set of other neurons from any layers become active or inactive, or have their bounds tightened. These implications can eliminate subproblems and improve bounds. We propose a scalable method to find implications among all neurons within tens of seconds even for large ResNets, by reusing pre-computed variables in popular bound-propagation-based verification methods such as $\alpha$-CROWN, and solving a cheap linear programming problem. Then, we build the bound implication graph (BIG) which connects neurons with bound implications, and it can be used by any BaB-based verifier to reduce the number of branching needed. When evaluated on a set of popular verification benchmarks and a new benchmark consisting of harder verification problems, BIG consistently reduces the verification time and verifies more problems than state-of-the-art verification tools.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8632