Rethinking the Solution to Curse of Dimensionality on Randomized Smoothing
Primary Area: societal considerations including fairness, safety, privacy
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Exponential Gaussian distribution, randomized smoothing, certified robustness, curse of dimensionality
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We find exponential general gaussian distributions provide a better solution to the curse of dimensionality in randomized smoothing, while perform greatly on real datasets in providing certified radius
Abstract: Randomized Smoothing (RS) is currently a scalable certified defense method providing robustness certification against adversarial examples.
Although significant progress has been achieved in providing defenses against $\ell_p$ adversaries,
early investigations found that RS suffers from the curse of dimensionality, indicating that the robustness guarantee offered by RS decays significantly with increasing input data dimension.
Double Sampling Randomized Smoothing (DSRS) is the state-of-the-art method that provides a theoretical solution to the curse of dimensionality under concentration assumptions on the base classifier.
However, we speculate the solution to the curse of dimensionality can be deepened from the perspective of the smoothing distribution.
In this work, we further address the curse of dimensionality by theoretically showing that some Exponential General Gaussian (EGG) distributions with the exponent $\eta$ can provide $\Omega(\sqrt{d})$ lower bounds for the $\ell_2$ certified radius with tighter constant factors than DSRS.
Our theoretical analysis shows that the lower bound improves with monotonically decreasing $\eta \in (0,2)$. Intriguingly, we observe a contrary phenomenon that EGG provides greater certified radii at larger $\eta$, on real-world tasks.
Further investigations show these discoveries are not contradictory, which are in essence dependent on whether the assumption in DSRS absolutely holds.
Our experiments on real-world datasets demonstrate that EGG distributions bring significant improvements for point-to-point certified accuracy, up to 4\%-6\% on ImageNet.
Furthermore, we also report the performance of Exponential Standard Gaussian (ESG) distributions on DSRS.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7661
Loading