Human Activity Recognition based on Collaboration of Vision and WiFi Signals

Shibo Li, Yao Ge, Minjian Shentu, Shuyuan Zhu, Muhammad Imran, Qammer H. Abbasi, Jonathan M. Cooper

Abstract: In the WiFi protocol, channel state information (CSI) is the modulated as the fine-grained data to assess the channel efficiency. Meanwhile, it contains the information about the environment change, including the movement of human in a specific environment. Therefore, the CSI data can be used to recognize the human activity. In this paper, we design a vision and WiFi collaboration-based human activity recognition scheme to classify the human activities. More specifically, we collect the CSI data from the WiFi signals and the human skeleton points from the video signals. Then, we construct a long-short-term Transformer network to build up the collaboration of the CSI data and the skeleton points. Based on this collaboration, we can use the CSI data to well recognize the human activities.