## Universal Approximation with Certified Networks

25 Sept 2019, 19:26 (modified: 11 Mar 2020, 07:33)ICLR 2020 Conference Blind SubmissionReaders: Everyone
Original Pdf: pdf
Code: https://github.com/eth-sri/UniversalCertificationTheory
Keywords: adversarial robustness, universal approximation, certified network, interval bound propagation
TL;DR: We prove that for a large class of functions f there exists an interval certified robust network approximating f up to arbitrary precision.
Abstract: Training neural networks to be certifiably robust is critical to ensure their safety against adversarial attacks. However, it is currently very difficult to train a neural network that is both accurate and certifiably robust. In this work we take a step towards addressing this challenge. We prove that for every continuous function \$f\$, there exists a network \$n\$ such that: (i) \$n\$ approximates \$f\$ arbitrarily close, and (ii) simple interval bound propagation of a region \$B\$ through \$n\$ yields a result that is arbitrarily close to the optimal output of \$f\$ on \$B\$. Our result can be seen as a Universal Approximation Theorem for interval-certified ReLU networks. To the best of our knowledge, this is the first work to prove the existence of accurate, interval-certified networks.
12 Replies