Human Action Recognition with Multi-Level Granularity and Pair-Wise Hyper GCN

Tamam Alsarhan, Syed Sadaf Ali, Ayoub Alsarhan, Iyyakutti Iyappan Ganapathi, Naoufel Werghi

Abstract: Lately, there has been a surge in interest in utilizing Graph Convolutional Networks (GCNs) for the purpose of action recognition using skeletal data. In order to achieve optimal results, it is crucial to generate high-quality representations of the skeletal graph. Graph Convolutional Networks (GCNs) often employ the Message-Passing Mechanism (MPM) to acquire knowledge about various components of the skeleton by iteratively computing new features at each step. However, the interconnections between joints in the skeletal structure are intricate and extend beyond mere proximity. In order to address this issue, we propose the implementation of our Disassembled Hyper-Graph (DH-Graph), which draws inspiration from hyper-graph edges. The process of constructing the DH-network entails a few steps: partitioning the skeleton network into clusters of hyper-edges according to their semantic significance and relevance to action recognition, arranging these clusters in a hierarchical structure to enhance granularity, and establishing connections between joints within these clusters to discover hidden relationships. The DH-Graph employs a spatial domain GCN technique to construct the Pair-wise Hyper Hierarchical GCN (PH-GCN). In addition, we incorporate the HyperAttention module, which employs Multi-scale Representative Spatial Average Pooling and Edge Convolution techniques to emphasize significant sets of hyper-hierarchical information. Extensive experiments demonstrate that PH-GCN achieves remarkable performance on challenging NTU RGB+D and Northwestern UCLA datasets.