Zero-shot Human Pose Estimation using Diffusion-based Inverse solvers

Sahil Bhandary Karnoor, Romit Roy Choudhury

Keywords: Human Motion tracking, Inverse problems, Wearable sensors

TL;DR: Our diffusion based algorithm performs 3-point pose estimation without needing any specific fine-tuning on a new user's body size through an inverse-guidance likelihood score.

Abstract:

Pose estimation refers to tracking a human's full body posture, including their head, torso, arms, and legs.
The problem is challenging in practical settings where the number of body sensors is limited.
Past work has shown promising results using conditional diffusion models, where the pose prediction is conditioned on both <location, rotation> measurements from the sensors.
Unfortunately, nearly all these approaches generalize poorly across users, primarily because location measurements are highly influenced by the body size of the user.
In this paper, we formulate pose estimation as an inverse problem and design an algorithm capable of zero-shot generalization.
Our idea utilizes a pre-trained diffusion model and conditions it on rotational measurements alone; the priors from this model are then guided by a likelihood term, derived from the measured locations.
Thus, given any user, our proposed InPose method generatively estimates the highly likely sequence of poses that best explains the sparse on-body measurements.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 10098