mmHPE: Human Pose Estimation Based on Point Cloud from Millimeter-wave Radar

Jiale Lai, Jiake Tian, Yi Zou, Xianfeng Song, Fangming Liu, Dacheng Li

Abstract: Rehabilitation therapy involving repetitive exer-cises targeting specific human joints under the supervision of a doctor is crucial for patients with movement disorders. Yet the cost of commuting and the demand for medical resources are inconvenient for patients. Human-computer interaction can provide remote rehabilitation guidance for patients at home through human pose estimation technology, but privacy concerns with optical sensors and the cost and discomfort of wearable sensors have hindered progress in this field. To address the above challenges, we propose mmHPE, an innovative 3D human pose estimation framework that uses millimeter-wave (mmWave) radar sensors. It initially manipulates the raw data captured from radar sensors to generate a spatiotemporal sequence point cloud dataset. Afterward, we create a Convolutional Neural Network (CNN) that is linked to a Bidirectional Long Short-Term Memory (Bi-LSTM) network. Moreover, a multi-head attention mechanism is employed to boost the network's performance and to accurately estimate the locations of human skeletons. Ultimately, the 21 points with the corre-sponding human pose position are successfully reconstructed. We investigate the mmHPE framework's feasibility and cross-domain stability in different home environments in real-world scenarios. This innovation proffers patients a convenient and privacy-conscious solution for their rehabilitation training req-uisites.