Hybrid two-stream dynamic CNN for view adaptive human action recognition using ensemble learning

Muhammad Hafeez Javed, Zeng Yu, Tianrui Li, Taha M. Rajeh, Fahad Rafique, Syed Waqar

Abstract: Human actions are sequential, and structured patterns of the body parts and their movements. In this paper, we present a hybrid two-stream convolutional neural network (H2SCNN) for the recognition of actions from sequences by exploring the statistical information like skeletons. This aims to exploit the skeletons completely and identify the actions properly by merging the different motion related features. These features include motion and joint features. The framework calculates the distance between consecutive sequences to form the temporal information required for the recognition process. The proposed H2SCNN is based on two stages. The neighbourhood feature model will be used to process both inputs individually in the first step. In the second stage, it performs ensemble learning and takes advantage of the diversity of multiple features by fusing them together. The multi-task ensemble learning model helps the system to improve the prediction ability of H2SCNN. Experiments on the benchmark dataset have shown the superiority of the proposed model with other recent approaches.