Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation

Maksym Yatsura, Kaspar Sakmann, N. Grace Hua, Matthias Hein, Jan Hendrik Metzen

Keywords: certified defences, adversarial robustness, adversarial patch attacks

TL;DR: We propose the first approach for certified recovery and certified detection against adversarial patch attacks on semantic segmentation, which is based on novel masking schemes and image inpainting.

Abstract: Adversarial patch attacks are an emerging security threat for real world deep learning applications. We present Demasked Smoothing, the first approach (up to our knowledge) to certify the robustness of semantic segmentation models against this threat model. Previous work on certifiably defending against patch attacks has mostly focused on image classification task and often required changes in the model architecture and additional training which is undesirable and computationally expensive. In Demasked Smoothing, any segmentation model can be applied without particular training, fine-tuning, or restriction of the architecture. Using different masking strategies, Demasked Smoothing can be applied both for certified detection and certified recovery. In extensive experiments we show that Demasked Smoothing can on average certify 63% of the pixel predictions for a 1% patch in the detection task and 46% against a 0.5% patch for the recovery task on the ADE20K dataset.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/certified-defences-against-adversarial-patch/code)