Slightly Harmonizing Certified Robust Radius and Accuracy

Gaojie Jin, Xinping Yi, Pengfei Yang, Xiaowei Huang, Lijun Zhang

Keywords: Certified robustness, Randomized smoothing, Generalization

Abstract: In the field of certified robustness through randomized smoothing, several works endeavor to improve the certified robust radius through, e.g., examining various smoothing distributions, conducting smooth training with adversarial data, or employing f-divergence based metrics. However, there is a lack of theoretical studies that delve into the relationship between the accuracy performance, the certified robust radius, and the model weights for smoothed classifiers. In the context of this study, we develop a generalization error bound that possesses a certified robust radius for a variant of the smoothed classifier (i.e., the classifier with both smoothed inputs and weights); In other words, the generalization bound holds under any data perturbation within the certified robust radius. As a byproduct, we find that the underpinnings of both the generalization bound and the certified robust radius draw, in part, upon weight spectral norm, which thereby inspires the adoption of spectral regularization in smooth training to boost certified robustness. Utilizing the dimension-independent property of spherical Gaussian inputs in smooth training, we propose a novel and inexpensive spectral regularizer to enhance smoothed classifiers. In addition to the theoretical contribution, an extensive set of empirical results is provided to substantiate the effectiveness of our proposed method.

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2572