Towards Calibrated Losses for Adversarial Robust Reject Option Classification

VRUND SHAH, Tejas Kiran Chaudhari, Naresh Manwani

Keywords: Calibrated Surrogates, Reject Option Classification, Adversarial Robustness

Verify Author List: I have double-checked the author list and understand that additions and removals will not be allowed after the submission deadline.

TL;DR: We provide a complete characterization of surrogates calibrated to the adversarial robust reject option loss and give insights on designing them.

Abstract: Robustness towards adversarial attacks is a vital property for classifiers in several applications such as autonomous driving, medical diagnosis, etc. Also, in such scenarios, where the cost of misclassification is very high, knowing when to abstain from prediction becomes crucial. A natural question is which surrogates can be used to ensure learning in scenarios where the input points are adversarially perturbed and the classifier can abstain from prediction? This paper aims to characterize and design surrogates calibrated in "Adversarial Robust Reject Option" setting. First, we propose an adversarial robust reject option loss $\ell_{d}^{\gamma}$ and analyze it for the hypothesis set of linear classifiers $\mathcal{H_\text{lin}}$. Next, we provide a complete characterization result for any surrogate to be $(\ell_{d}^{\gamma},\mathcal{H_{\text{lin}}})$- calibrated. To demonstrate the difficulty in designing surrogates to $\ell_{d}^{\gamma}$, we show negative calibration results for convex surrogates and quasi-concave conditional risk cases (these gave positive calibration in adversarial setting without reject option). We also empirically argue that Shifted Double Ramp Loss (DRL) and Shifted Double Sigmoid Loss (DSL) satisfy the calibration conditions. Finally, we demonstrate the robustness of shifted DRL and shifted DSL against adversarial perturbations on a synthetically generated dataset.

Url Link To Your Supplementary Code: https://github.com/Vrund0212/Calibrated-Losses-for-Adversarial-Robust-Reject-Option-Classification

Primary Area: Theory (bandits, computational learning theory, game theory, optimization, statistical learning theory, etc.)

Paper Checklist Guidelines: I certify that all co-authors of this work have read and commit to adhering to the guidelines in Call for Papers.

Student Author: Yes

Submission Number: 352