Reducing Certified Regression to Certified Classification for General Poisoning Attacks

Zayd Hammoudeh, Daniel Lowd

Keywords: Robust regression, Certified classifier, Data poisoning, Partial set cover, Partial set multicover

TL;DR: We propose a simple reduction from certified regression to certified classification, and coupling our reduction with existing certified classifiers, we propose six new provably-robust regressors.

Abstract: Adversarial training instances can severely distort a model’s behavior. This work investigates certified regression defenses, which provide guaranteed limits on how much a regressor’s prediction may change under a training-set attack. Our key insight is that certified regression reduces to certified classification when using median as a model’s primary decision function. Coupling our reduction with existing certified classifiers, we propose six new provably-robust regressors. To the extent of our knowledge, this is the first work that certifies the robustness of individual regression predictions without any assumptions about the data distribution and model architecture. We also show that the assumptions made by existing state-of-the-art certified classifiers are often overly pessimistic. We introduce a tighter analysis of model robustness, which in many cases results in significantly improved certified guarantees. Lastly, we empirically demonstrate our approaches’ effectiveness on both regression and classification data, where the accuracy of up to 50% of test predictions can be guaranteed under 1% training-set corruption and up to 30% of predictions under 4% corruption. Our source code is available at https://github.com/ZaydH/certified-regression.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/reducing-certified-regression-to-certified/code)