Physics-Augmented Autoencoder for 3D Skeleton-Based Gait Recognition

Hongji Guo, Qiang Ji

Abstract:

In this paper, we introduce physics-augmented autoencoder (PAA) framework for 3D skeleton-based human gait
recognition. Specifically, we construct the autoencoder with
a graph-convolution-based encoder and a physics-based
decoder. The encoder takes the skeleton sequence as input and produces the generalized positions and forces of
each joint, which are taken by the decoder to reconstruct the
input skeleton based on the Lagrangian dynamics. In this
way, the intermediate representations are physically plausible and discriminative. During the inference, the decoder is
discared and a RNN-based classifier takes the output of the
encoder for gait recognition. We evaluated our proposed
method on three benchmark datasets including Gait3D,
GREW, and KinectGait. Our method achieves state-ofthe-art performance for 3D skeleton-based gait recognition. Furthermore, extensive ablation studies show that our
method generalizes better and is more robust with smallscale training data by incorporating the physics knowledge.
We also validated the physical plausibility of the intermediate representations by making force predictions on real data
with physical annotations.