Multi-task learning for gait-based identity recognition and emotion recognition using attention enhanced temporal graph convolutional network

Weijie Sheng, Xinde Li

Abstract: Highlights•To our knowledge, this is the first work to treat identity recognition and emotion recognition as related tasks for jointly learning. We propose a multi-task learning architecture for gait-related recognition problems and achieve better performances by sharing knowledge.•We propose a novel AT-GCN network for gait skeleton sequences, which can effectively capture discriminative spatiotemporal gait features. The attention mechanism is employed to enhance the expressive capability for achieving higher performance.•We present a new dataset of human gaits (EMOGAIT), which consists of 1, 440 real-world gait videos annotated with identity labels and emotion labels. The proposed model achieves state-of-the-art results on both EMOGAIT dataset and TUMGAID dataset.