SPACE: Speaker Adaptation for Acoustic Eavesdropping Using mmWave Radio Signals

Running Zhao, Luca Jiang-Tao Yu, Tingle Li, Zhihan Jiang, Chenwei Zhang, Chenshu Wu, Hang Zhao, Edith C.H. Ngai

Abstract: The prevalence of voice-related interaction and communication has raised concerns about privacy leakage and security. For example, millimeter-wave (mmWave) radio signals have been exploited as a potential attacker for acoustic eavesdropping. However, speaker variability and low-quality input pose significant challenges for the practical deployment of mmWave-based eavesdropping. In this paper, we propose SPACE, an acoustic eavesdropping system to recover intelligible speech from low-quality mmWave signals, which can adapt to numerous different speakers and unseen ones. SPACE is a two-stage system that first reconstructs the spectrogram using a novel Radio TransUNet and then synthesizes the waveform through a neural vocoder. Specifically, to alleviate the negative effect of speaker variability, we introduce a speaker encoder to capture speaker features and a fusion network to condition the spectrogram reconstruction based on the extracted speaker characteristics. Further, to facilitate intelligible speech recovery from low-quality input, we design a Frequency Transformation Layer to exploit the correlation among all frequency harmonics and incorporate the neural vocoder to synthesize the speech waveform from the reconstructed spectrogram without using the contaminated phase. The experimental results show that SPACE outperforms existing mmWave-based approaches in scenarios with numerous different speakers and unseen speakers.

External IDs:doi:10.1109/tmc.2025.3598703