Data Poisoning Won’t Save You From Facial RecognitionDownload PDF

Published: 21 Jun 2021, Last Modified: 05 May 2023ICML 2021 Workshop AML OralReaders: Everyone
Keywords: Poisoning attacks, facial recognition, arms race, adaptive defenses
TL;DR: Data poisoning attacks against facial recognition systems face a losing battle and provide users with a false sense of security.
Abstract: Data poisoning has been proposed as a compelling defense against facial recognition models trained on Web-scraped pictures. By perturbing the images they post online, users can fool models into misclassifying future (unperturbed) pictures. We demonstrate that this strategy provides a false sense of security, as it ignores an inherent asymmetry between the parties: users' pictures are perturbed once and for all before being published and scraped, and must thereafter fool all future models---including models trained adaptively against the users' past attacks, or models that use technologies discovered after the attack. We evaluate two poisoning attacks against large-scale facial recognition, Fawkes 500,000+ downloads) and LowKey. We demonstrate how an ``oblivious'' model trainer can simply wait for future developments in computer vision to nullify the protection of pictures collected in the past. We further show that an adversary with black-box access to the attack can train a robust model that resists the perturbations of collected pictures. We caution that facial recognition poisoning will not admit an ''arms race'' between attackers and defenders. Once perturbed pictures are scraped, the attack cannot be changed so any future defense irrevocably undermines users' privacy.
2 Replies

Loading