Deep Hyperspherical Defense against Adversarial Perturbations

Weiyang Liu, Zhen Liu, Zhehui Chen, Bo Dai, Tuo Zhao, Le Song

Abstract: Recent studies show that deep neural networks are extremely vulnerable to adversarial examples which are semantically indistinguishable from natural data and yet incorrectly classified. These adversarial examples are generated from the natural data by adding a small amount of adversarial perturbation. This paper tackles the adversarial attack problem with hyperspherical defense - a defense strategy that learns neural network over hyperspheres. The hyperspherical defense framework is well motivated by: (i) Learning on hyperspheres gives us bounded output, which may make the geometry of neural networks more smooth; (ii) Learning on hyperspheres could naturally eliminate some adversarial perturbations and reduce the effect of adversarial perturbations; (iii) Representing data on hyperspheres selectively drops some information of the inputs, but these information are shown to be not crucial to visual recognition (based on the fact that hyperspherical neural network performs comparable to or even better than standard neural networks in visual recognition). Furthermore, we introduce the hyperspherical compactness and propose a robust geodesic inference. We also provide theoretical insights about why our hyperspherical defense improves robustness. Last, we validate the superiority of hyperspherical defense with extensive experiments on both white-box and black-box adversarial attacks on multiple datasets.