Neural Network Branching for Neural Network Verification

Jingyue Lu, M. Pawan Kumar

Keywords: Neural Network Verification, Branch and Bound, Graph Neural Network, Learning to branch

TL;DR: We propose a novel learning to branch framework using graph neural networks to improve branch and bound based neural network verification methods.

Abstract: Formal verification of neural networks is essential for their deployment in safety-critical areas. Many available formal verification methods have been shown to be instances of a unified Branch and Bound (BaB) formulation. We propose a novel framework for designing an effective branching strategy for BaB. Specifically, we learn a graph neural network (GNN) to imitate the strong branching heuristic behaviour. Our framework differs from previous methods for learning to branch in two main aspects. Firstly, our framework directly treats the neural network we want to verify as a graph input for the GNN. Secondly, we develop an intuitive forward and backward embedding update schedule. Empirically, our framework achieves roughly $50\%$ reduction in both the number of branches and the time required for verification on various convolutional networks when compared to the best available hand-designed branching strategy. In addition, we show that our GNN model enjoys both horizontal and vertical transferability. Horizontally, the model trained on easy properties performs well on properties of increased difficulty levels. Vertically, the model trained on small neural networks achieves similar performance on large neural networks.

Code: [ oval-group/GNN_branching](https://github.com/oval-group/GNN_branching)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1912.01329/code)