Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain

Gaozheng Pei, Ke Ma, Yingfei Sun, Qianqian Xu, Qingming Huang

TL;DR: Adversarial Purification from Perspective of Frequency Domain

Abstract:

The diffusion-based adversarial purification methods attempt to drown adversarial perturbations into a part of isotropic noise through the forward process, and then recover the clean images through the reverse process. Due to the lack of distribution information about adversarial perturbations in the pixel domain, it is often unavoidable to damage normal semantics. We turn to the frequency domain perspective, decomposing the image into amplitude spectrum and phase spectrum. We find that for both spectra, the damage caused by adversarial perturbations tends to increase monotonically with frequency. This means that we can extract the content and structural information of the original clean sample from the frequency components that are less damaged. Meanwhile, theoretical analysis indicates that existing purification methods indiscriminately damage all frequency components, leading to excessive damage to the image. Therefore, we propose a purification method that can eliminate adversarial perturbations while maximizing the preservation of the content and structure of the original image. Specifically, at each time step during the reverse process, for the amplitude spectrum, we replace the low-frequency components of the estimated image's amplitude spectrum with the corresponding parts of the adversarial image.
For the phase spectrum, we project the phase of the estimated image into a designated range of the adversarial image's phase spectrum, focusing on the low frequencies. Empirical evidence from extensive experiments demonstrates that our method significantly outperforms most current defense methods.

Lay Summary: We propose a new method to counteract adversarial attacks on images. Traditional methods attempt to remove adversarial noise, but often end up damaging important features of the image. Our approach looks at images in the frequency domain (focusing on the image’s frequency components, such as the amplitude and phase spectra). We find that adversarial noise increases with frequency, so by concentrating on the less affected low-frequency components, we can better preserve the original content and structure of the image. We improve the purification process by carefully modifying these low-frequency parts, both in terms of amplitude and phase spectra. Extensive experiments show that our method significantly outperforms existing defense methods.

Link To Code: https://github.com/GaozhengPei/FreqPure

Primary Area: Social Aspects->Security

Keywords: Adversarial Purification

Submission Number: 1502