Human Action Categorization System using Body Pose Estimation for Multimodal Observations from Single Camera

Aditya Singh Rathore, Lotfi El Hafi, Gustavo Alfonso Garcia Ricardez, Tadahiro Taniguchi

Abstract: We propose a system using a multimodal probabilistic approach to solve the human action recognition challenge. This is achieved by extracting the human pose from an ongoing activity from a single camera. This pose is used to capture additional body information using generalized features such as location, time, distances, and angles. A probabilistic model, multimodal latent Dirichlet allocation (MLDA), which uses this multimodal information, is then used to recognize actions through topic modeling. We also investigate the influence of each modality and their combinations to recognize human actions from multimodal observations. The experiments show that the proposed generalized features captured significant information that enabled the classification of various daily activities without requiring prior labeled data.