Structural Attention for Channel-Wise Adaptive Graph Convolution in Skeleton-Based Action Recognition

Ruihao Qian, Jiewen Wang, Jianxiu Wang, Shuang Liang

Abstract: In skeleton-based action recognition, graph convolutions to model human action dynamics have been widely implemented and achieved remarkable results. Among these convolutions, channel-wise adaptive graph convolution shows outstanding performance. However, this method focuses too much on capturing correlation between joints within each channel and lacks the capability of learning structural features, which are generally hidden in geometric property of the skeleton on spatial domain. Our proposed method (SA-GCN) introduces symmetry trajectory attention module to measure the relation between left and right part of body and part relation attention module for exploration of the attention on general relation of each part. Both modules are intended to make full use of structural features in skeleton, further strengthening advantages of graph convolution. Experiments on three datasets (NW-UCLA, NTU-RGB+D and NTU-RGB+D 120) demonstrate state-of-the-art performance of our model, especially on joint modality.