PatchBreaker: defending against adversarial attacks by cutting-inpainting patches and joint adversarial training

Shiyu Huang, Feng Ye, Zuchao Huang, Wei Li, Tianqiang Huang, Liqing Huang

Abstract: Adversarial patches can disrupt computer vision systems, seriously threatening people’s lives and property security. Existing defense methods seldom consider the generalization for defending against different patches and the compatibility with various models. Furthermore, the severe security situation necessitates the combination of data defense and model defense to build a comprehensive defense system. To address these issues, we propose a defense method named PatchBreaker, which consists of three components. In data defense, PatchBreaker uses the Semantic-Cutter trained by annotated patch images to cut patches and output incomplete images. Next, the Image-Inpainter trained by clean-incomplete image pairs is used to inpaint these incomplete images and output inpainted images. In model defense, the Adversaril-Classifier will be trained by joint adversarial training with clean images and patch images. Finally, PatchBreaker inputs inpainted images into Adversarial-Classifier to output correct results. Comparative experiments show that PatchBreaker outperforms other comparative defense methods in most cases, which indicates the excellent patch generalization and model compatibility of PatchBreaker. Meanwhile, ablation studies show the effectiveness of combining data defense and model defense. Additionally, PatchBreaker has minimal impact on the clean accuracy (about 1\(\%\)).