Diffusion-based Adversarial Attack to Automatic Speech Recognition

Ying Wang, Yuchuan Luo, Shaojing Fu, Zhenyu Qiu, Lin Liu

Keywords: Trustworthy Machine Learning; Adversarial Examples; Deep Learning; Automatic Speech Recognition; Voice Conversion

Verify Author List: I have double-checked the author list and understand that additions and removals will not be allowed after the submission deadline.

TL;DR: Enhancing Acoustic Realism and Adversarial Performance of Audio Adversarial Examples on Automatic Speech Recognition model througn Diffusion-based Voice Conversion

Abstract: Recent studies have exposed the substantial vulnerability of voice-activated smart devices to adversarial examples, predominantly targeting the robustness of automatic speech recognition (ASR) systems. Most of adversarial examples generated by introducing adversarial perturbations within the $l_p$ norm bounds to benign audio inputs. However, these attacks are constrained by the parametric bounds of perturbations or the features of disturbance, which limits their effectiveness. To improve the acoustic realism of adversarial examples and enhance attack performance, we propose a novel attack framework called Diffusion-based Adversarial Attack, leveraging DiffVC, a diffusion-based voice conversion model, to map audio to a latent space and employing Adversarial Latent Perturbation (ALP) to embed less perceptible and more robust perturbations. Extensive evaluations demonstrate that our method enhances targeted attack performance. Notably, the Word Error Rate (WER) has shown an average increase of 101 absolute points over clean speech audio and 25 absolute points over C\&W attack. Additionally, the Success Rate (SR) has achieved an average increase of 11 absolute points over the C\&W attack and 16 absolute points over SSA attack. Additionally, our approach also stands out for its high audio quality and efficiency.

Primary Area: Trustworthy Machine Learning (accountability, explainability, transparency, causality, fairness, privacy, robustness, autoML, etc.)

Paper Checklist Guidelines: I certify that all co-authors of this work have read and commit to adhering to the guidelines in Call for Papers.

Student Author: Yes

Submission Number: 265