TWLBR: Multi-Human Through-Wall Localization and Behavior Recognition Based on MIMO Radar

Dongsheng Zhu, Changlong Wang, Chong Han, Jian Guo, Lijuan Sun

Abstract: Human localization and behavior recognition (HLBR) is an important research topic in wireless sensing and computer vision. In existing work, most of the approaches using sensors such as cameras and mmWave radar cannot solve the wall occlusion problem, while the approaches using Wi-Fi can penetrate the wall but cannot locate human precisely due to its bandwidth limitation. All these limit the application of HLBR in reality. In this paper, we propose TWLBR, a real-time detection system for inferring the localization and behavior of human behind brick walls from radar heatmaps. In this system, we design a multiple-input multiple-output (MIMO) radar in the frequency range of 1–2 GHz and a multi-feature fusion network based on a 3D convolutional neural network (CNN) and a transformer. The network takes four radar heatmaps with background removal as input and outputs the location and behavior of the target humans. Our experiments show that TWLBR can locate and recognize the behavior of target humans behind a 24 cm brick wall with a localization accuracy of 6.2 cm and a behavior recognition accuracy of 96.37%, which is better than existing methods.