MHTrack: mmWave-Based Mobile Hand Tracking

Xiulong Liu, Hankai Liu, Yantao Han, Xin Xie, Xinyu Tong, Keqiu Li

Abstract: Non-intrusive hand tracking with mmWave radar technology is important in various Human-Computer Interaction (HCI) scenarios. However, existing mmWave-based solutions require users to be stationary and restrict a fixed hand motion area, which limits application flexibility and user experience. This paper proposes a novel mmWave-based Mobile Hand Tracking (MHTrack) system, which tracks user’s hand gestures during walking. MHTrack focuses on tracking both absolute hand trajectory in the global coordinate system and relative hand trajectory to the body. Specifically, we propose a wake-up mechanism for hand motion capture, in which hand point cloud can be recognized even under body interference and noise. We propose a hand tracking strategy named local spatial update, which overcomes the sparsity and instability of point clouds, to obtain absolute hand trajectory. Subsequently, we propose a hand anchor correction method to suppress anchor offset and remove the impact of body movement from absolute hand trajectory, thereby obtaining relative hand trajectory. As a case study, we project the relative hand trajectory onto a 2D image and feed it into a gesture recognition model to recognize the gestures. We conduct extensive experiments to evaluate the performance of MHTrack. Results demonstrate a 3D hand trajectory tracking error of $ 3.6$ cm in an area of $ 3.2\;{\rm m}\,\times\, 4.8\;{\rm m}$ and a gesture recognition accuracy of 99% with 30 gesture classes.

External IDs:dblp:journals/tmc/LiuLHXTL25