Confidence-aware Training of Smoothed Classifiers for Certified Robustness

Jongheon Jeong, Seojin Kim, Jinwoo Shin

Keywords: randomized smoothing, adversarial robustness, certified defense, adversarial defense, robust training, confidence calibration

Abstract: Any classifier can be "smoothed out" under Gaussian noise to build a new classifier that is provably robust to $\ell_2$-adversarial perturbations, viz., by averaging its predictions over the noise, namely via randomized smoothing. Under the smoothed classifiers, the fundamental trade-off between accuracy and (adversarial) robustness has been well evidenced in the literature: i.e., increasing the robustness of a classifier for an input can be at the expense of decreased accuracy for some other inputs. In this paper, we propose a simple training method leveraging this trade-off for obtaining more robust smoothed classifiers, in particular, through a sample-wise control of robustness over the training samples. We enable this control feasible by investigating the correspondence between robustness and prediction confidence of smoothed classifiers: specifically, we propose to use the "accuracy under Gaussian noise" as an easy-to-compute proxy of adversarial robustness for each input. We differentiate the training objective depending on this proxy to filter out samples that are unlikely to benefit from the worst-case (adversarial) objective. Our experiments following the standard benchmarks consistently show that the proposed method, despite its simplicity, exhibits improved certified robustness upon existing state-of-the-art training methods.

One-sentence Summary: We propose a novel training objective for randomized smoothing with state-of-the-art results in certified robustness, leveraging the relationship between confidence and robustness of smoothed classifiers.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2212.09000/code)