ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation

Cédric Rommel, Victor Letzelter, Nermin Samet, Renaud Marlet, Matthieu Cord, Patrick Perez, Eduardo Valle

Keywords: human pose estimation, multiple choice learning, manifold estimation

TL;DR: We prove previous 2D-to-3D human pose lifting methods suffer from topology inconsistencies invisible to standard evaluation metrics, and propose a new method circumventing these issues via constrained multiple hypotheses.

Abstract: We provide theoretical and empirical evidence that, due to the depth ambiguity inherent to monocular 3D human pose estimation, traditional regression models suffer from pose-topology consistency issues, which standard evaluation metrics (MPJPE, P-MPJPE and PCK) fail to assess. We hence propose ManiPose, a manifold-constrained multi-hypothesis model for human-pose 2D-to-3D lifting. ManiPose addresses depth ambiguity by proposing multiple candidate 3D poses for each 2D input, each with its estimated plausibility. By constraining the outputs to lie on the human pose manifold, ManiPose guarantees the consistency of all hypothetical poses. We showcase the performance of ManiPose on simulated and real-world datasets, where it outperforms state-of-the-art models in pose consistency by a large margin while being very competitive on the MPJPE metric.

Submission Number: 11