Benchmarking the Effect of Poisoning Defenses on the Security and Bias of the Final Model

Nathalie Baracaldo, Kevin Eykholt, Farhan Ahmed, Yi Zhou, Shriti Priya, Taesung Lee, Swanand Kadhe, Yusong Tan, Sridevi Polavaram, Sterling Suggs, Yuyang Gao, David Slater

Keywords: Poisoning attacks, security, machine learning

TL;DR: We propose a new metric to characterize poisoning defenses to measure their side effects and benchmark existing defenses with this new metric as well as traditional security metrics; our metric can provide further information to contrast defenses.

Abstract: Machine learning models are susceptible to a class of attacks known as adversarial poisoning where an adversary can maliciously manipulate training data to hinder model performance or, more concerningly, insert backdoors to exploit at inference time. Many methods have been proposed to defend against adversarial poisoning by either identifying the poisoned samples to facilitate removal or developing poison agnostic training algorithms. Although effective, these proposed approaches can have unintended consequences on other aspects of model performance, such as worsening performance on certain data sub-populations, thus inducing a classification bias. In this work, we evaluate several adversarial poisoning defenses. In addition to traditional security metrics, i.e., robustness to poisoned samples, we propose a new metric to measure the potential undesirable discrimination of sub-populations resulting from using these defenses. Our investigation highlights that many of the evaluated defenses trade decision fairness to achieve higher adversarial poisoning robustness. Given these results, we recommend our proposed metric to be part of standard evaluations of machine learning defenses.