Privacy-preserving Human Activity Recognition via Video-based Range-Doppler Synthesis

Zhiyuan Cui, Luoyu Mei, Siyuan Pei, Borui Li, Xiaolei Zhou

Abstract: As an important branch of IoT applications, Human activity recognition (HAR) is widely used in daily life, particularly through vision-based methods. However, vision-based HAR has serious privacy issues. How to better and low-cost protect the privacy of users who have already installed the relevant devices is a problem that needs to be solved. To address this challenge, we can solve it by transforming video to privacy-preserving mmWave data. Existing studies have primarily focused on synthesizing micro-Doppler data from video, but there is a lack of methods for synthesizing range-Doppler data. Thus, we present a comprehensive method for synthesizing range-Doppler data from videos and subsequently utilize this synthetic data for HAR. Experimentally, we deploy our range-Doppler synthesis method and classification model on a custom dataset. Experimental results indicate that the model trained with synthetic data achieves accuracy on the custom dataset by 95.7%, which is comparable to the accuracy of vision-based HAR works, and demonstrate that the scheme proposed in this paper achieves privacy-preserving HAR.