Transforming spatio-temporal self-attention using action embedding for skeleton-based action recognition

Tasweer Ahmad, Syed Tahir Hussain Rizvi, Neel Kanwal

Abstract: Highlights•Action recognition has several computer applications such as surveillance, content analysis, healthcare, etc.•We have proposed a new action embedding scheme using a graph convolution and link prediction method that can help exploit the relation between distant joints.•We tune the embeddings using a self-attention transformer to preserve longer spatial–temporal dependencies across frames for final multiclass classification.•We have evaluated our method of different public datasets, including SYSU-3D, NTU RGB+D, and NTU RGB+D 120, where it surpasses state-of-the-art methods.