A Little Is Enough: Circumventing Defenses For Distributed Learning

Gilad Baruch, Moran Baruch, Yoav Goldberg

Abstract: Distributed learning is central for large-scale training of deep-learning models. However, they are exposed to a security threat in which Byzantine participants can interrupt or control the learning process. Previous attack models assume that the rogue participants are (a) omniscient (know the data of all other participants), and (b) introduce large change to the parameters. Accordingly, defense mechanisms make a similar assumption and attempt to identify and discard values whose reported gradients are far from the population mean. We observe that if the empirical variance between the gradients of workers is high enough, an attacker could take advantage of this and launch a non-omniscient attack that operates within the population variance. We show that the variance is indeed high enough even for simple models such as MNIST, allowing an attack that is not only undetected by existing state of the art defenses, but also uses their power against them, causing the defense mechanisms to consistently select the byzantine workers while discarding legitimate ones. This makes defended models more vulnerable than vanilla ones. We demonstrate our attack method works not only for preventing convergence but also for repurposing of the model behavior (``backdooring''). We show that less than 25% of corrupt workers are sufficient to degrade MNIST, CIFAR10 and CIFAR100 models' accuracy by 50%, as well as to introduce backdoors without hurting the accuracy for MNIST and CIFAR10 models, but with a degradation for CIFAR100.

Code Link: https://github.com/moranant/attacking_distributing_learning

CMT Num: 4657