SONAR: Spectral‑Contrastive Audio Residuals for Robust Deepfake Detection

Ido Nitzan Hidekel, Gal Lifshitz, Khen Cohen, Dan Raviv

Keywords: Frequency learning, audio deepfake detection, speech, spectral bias

TL;DR: We introduce SONAR, a dual path network that overcomes spectral bias in audio DF detection by disentangling and contrastively aligning low and high frequency features.

Abstract: Deepfake (DF) audio detectors still struggle to generalize to out of distribution inputs. A central reason is \emph{spectral bias}, the tendency of neural networks to learn low‑frequency structure before high‑frequency (HF) details, which both causes DF generators to leave HF artifacts and leaves those same artifacts under-exploited by common detectors. To address this gap, we propose \textbf{Spectral‑cONtrastive Audio Residuals (SONAR)}, a frequency‑guided framework that explicitly disentangles an audio signal into complementary representations. An XLSR encoder captures the dominant low‑frequency content, while the same cloned path, preceded by learnable SRM, value-constrained high‑pass filters, distills faint HF residuals. Frequency cross‑attention reunites the two views for long‑ and short‑range frequency dependencies, and a frequency‑aware Jensen–Shannon contrastive loss pulls real content–noise pairs together while pushing fake embeddings apart, accelerating optimization and sharpening decision boundaries. Evaluated on the ASVspoof 2021 and in‑the‑wild benchmarks, SONAR attains state‑of‑the‑art performance and converges four times faster than strong baselines. By elevating faint high‑frequency residuals to first‑class learning signals, SONAR unveils a fully data‑driven, frequency‑guided contrastive framework that splits the latent space into two disjoint manifolds: natural‑HF for genuine audio and distorted‑HF for synthetic audio, thereby sharpening decision boundaries. Because the scheme operates purely at the representation level, it is architecture‑agnostic and, in future work, can be seamlessly integrated into any model or modality where subtle high‑frequency cues are decisive.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 7170