Exact and Efficient Adversarial Robustness with Decomposable Neural Networks

Pranav Shankar Subramani, antonio vergari, Gautam Kamath, Robert Peharz

Keywords: Adversarial Robustness, Certifiable Guarantees, Non-Probabilistic Circuits, Deep Learning

TL;DR: Certification of adversarially robust inputs requires Monte-Carlo evaluation which we circumvent with a new architecture.

Abstract:

As deep neural networks are notoriously vulnerable to adversarial attacks, there has been significant interest in defenses with provable guarantees.
Recent solutions advocate for a randomized smoothing approach to provide probabilistic guarantees, by estimating the expectation of a network's output when the input is randomly perturbed.
As the convergence of the estimated expectations depends on the number of Monte Carlo samples, and hence network evaluations, these techniques come at the price of considerable additional computation at inference time.
We take a different route and introduce a novel class of deep models---decomposable neural networks (DecoNets)---which compute the required expectation \textit{exactly} and efficiently using a \textit{single network evaluation}.
This remarkable feature of DecoNets stems from their network structure, implementing a hierarchy of \textit{decomposable multiplicative interactions} over non-linear input features, which allows to reduce the overall expectation into many ``small'' expectations over input units, thus delivering \textit{exact guarantees}.