Reconstructing Missing Joints in 3D Human Motion with Temporal-Structural Awareness Graph Neural Network

Minh Duc Nguyen, Thi Linh Hoang, Viet Cuong Ta

Abstract: In human motion capture-based applications, the 3D captured pose data usually contains missing joints due to various factors such as environmental conditions, camera quality, and joint occlusions. Since the 3D pose data can be represented as a sequence of graphs, Graph Neural Networks can be used to effectively capture the complex and dynamic relationships among the nodes in the data. In this paper, we model the pose sequence of human motion as a dynamic graph with missing nodes' positions in 3D coordinates and reconstruct it by exploiting the temporal and structural properties of the input graph. Based on this modeling, we propose the TA-WLS architecture, which utilizes concatenate aggregation to preserve the local structural information and an attention mechanism to effectively capture the temporal relations among nodes. The proposed TA-WLS is tested on the Human 3.6M dataset under various missing conditions, and the results show that our TA-WLS can significantly outperform other baselines by a large margin.