Accumulative Poisoning Defense with Memorization Discrepancy

Jianing Zhu, Xiawei Guo, Jiangchao Yao, Chao Du, LI He, Shuo Yuan, Tongliang Liu, Liang Wang, Bo Han

Abstract: Adversarial poisoning attacks pose huge threats to various machine learning applications. Especially, the recent accumulative poisoning attacks show that it is possible to achieve irreparable harm on models via a sequence of imperceptible attacks followed by the trigger sample. Due to the limited data-level information in real-time data streaming, the current defensive methods are indiscriminate in handling the poison and clean samples. In this paper, we dive into the perspective of model dynamics and propose a novel information measure, namely, Memorization Discrepancy, to explore the defense via the model-level information. Through implicitly transferring changes in the data manipulation to that in model outputs, our Memorization Discrepancy constructed by the victim and historical models is aware of the imperceptible poison samples based on their distinct values from the clean samples. We thoroughly analyze its properties and accordingly propose a Discrepancy-aware Sample Correction (DSC) to defend against the accumulative poisoning attacks. Extensive experiments comprehensively characterize our proposed Memorization Discrepancy and verified the effectiveness of our DSC.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning