Towards generalizing deep-audio fake detection networks

Published: 08 Apr 2024, Last Modified: 08 Apr 2024Accepted by TMLREveryoneRevisionsBibTeX
Abstract: Today's generative neural networks allow the creation of high-quality synthetic speech at scale. While we welcome the creative use of this new technology, we must also recognize the risks. As synthetic speech is abused for monetary and identity theft, we require a broad set of deepfake identification tools. Furthermore, previous work reported a limited ability of deep classifiers to generalize to unseen audio generators. We study the frequency domain fingerprints of current audio generators. Building on top of the discovered frequency footprints, we train excellent lightweight detectors that generalize. We report improved results on the WaveFake dataset and an extended version. To account for the rapid progress in the field, we extend the WaveFake dataset by additionally considering samples drawn from the novel Avocodo and BigVGAN networks. For illustration purposes, the supplementary material contains audio samples of generator artifacts.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: N/A
Supplementary Material: zip
Assigned Action Editor: ~Xu_Tan1
Submission Number: 1969