Distillation based Robustness Verification with PAC Guarantees

Patrick Indri, Peter Blohm, Anagha Athavale, Ezio Bartocci, Georg Weissenbacher, Matteo Maffei, Dejan Nickovic, Thomas Gärtner, SAGAR MALHOTRA

Keywords: Robustness, Knowledge Distillation, Formal Verification, PAC-Verification

TL;DR: We distill a large NN into a smaller NN that can be tractably checked for local-robustness. We provide sample complexity for the distillation procedure that certifies the global robustness of the teacher NN w.h.p.

Abstract: We present a distillation based approach to verify the robustness of any Neural Network (NN). Conventional formal verification methods cannot tractably assess the global robustness of real-world NNs. To address this, we take advantage of a gradient-aligned distillation framework to transfer the robustness properties from a larger teacher network to a smaller student network. Given that the student NN can be formally verified for global robustness, we theoretically investigate how this guarantee can be transferred to the teacher NN. We draw from ideas in learning theory to derive a sample complexity for the distillation procedure that enables PAC-guarantees on the global robustness of the teacher network.

Submission Number: 85