A multi-graph convolutional network based wearable human activity recognition method using multi-sensors

Ling Chen, Yingsong Luo, Liangying Peng, Rong Hu, Yi Zhang, Shenghuan Miao

Abstract: Wearable human activity recognition (WHAR) using multi-sensors is a promising research area in ubiquitous and wearable computing. Existing WHAR methods usually interact features learned from multi-sensor data by using convolutional neural networks or fully connected networks, which may ignore the prior relationships among multi-sensors. In this paper, we propose a novel method, called MG-WHAR, which employs graphs to model the relationships among multi-sensors. Specifically, we construct three types of graphs: a body structure based graph, a sensor modality based graph, and a data pattern based graph. In each graph, the nodes represent sensors, and the edges are set according to the relationships among sensors. MG-WHAR, utilizing a multi-graph convolutional network, conducts feature interactions by leveraging the relationships among multi-sensors. This strategy not only enhances model performance but also results in a model with fewer parameters. Compared to the state-of-the-art WHAR methods, our method increases weighted F1-score by 3.2% on Opportunity dataset, 1.9% on Realdisp dataset, and 2.6% on DSADS dataset, while maintaining lower computational complexity.