Formal Verification for Neural Networks with General Nonlinearities via Branch-and-Bound
Keywords: neural network verification, formal verification, robustness verification
Abstract: Bound propagation with branch-and-bound (BaB) is so far among the most effective methods for neural network (NN) verification. However, existing works with BaB have mostly focused on NNs with piecewise linear activations, especially ReLU networks. In this paper, we develop a framework for conducting BaB based on bound propagation with general branching points and an arbitrary number of branches, as an important move for extending NN verification to models with various nonlinearities beyond ReLU. Our framework strengthens verification for common element-wise activation functions, as well as other multi-dimensional nonlinear operations such as multiplication. In addition, we find that existing heuristics for choosing neurons to branch for ReLU networks are insufficient for general nonlinearities, and we design a new heuristic named BBPS, which usually outperforms the heuristic obtained by directly extending the existing ones originally developed for ReLU networks. We empirically demonstrate the effectiveness of our BaB framework on verifying a wide range of NNs, including networks with Sigmoid, Tanh, sine or GeLU activations, LSTMs and ViTs, which have various nonlinearities. Our framework also enables applications with models beyond neural networks, such as models for AC Optimal Power Flow (ACOPF).
Primary Area: societal considerations including fairness, safety, privacy
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6011
Loading