Low-latency online estimation of human upper-limb pose and kinematics from a single 360° camera

Mathis D'Haene, Guillaume Caron, Yusuke Yoshiyasu, Bruno Watier

Abstract: We present a fully online framework for streaming human upper-limb kinematics estimation from a single 360° camera. Incoming frames are processed sequentially through vertical-boundary-aware tracking, pseudo-perspective rendering, and Neural Localizer Fields to estimate a sparse set of 3D anatomical landmarks in real time. These landmarks are mapped to an OpenSim-compatible biomechanical model, with joint angles computed on the fly via an online inverse kinematics solver. The system achieves end-to-end latencies as low as 22.9 ms on a high-performance setup. Evaluated in a single-participant scenario involving an initial T-pose calibration and repeated object displacement toward the camera, it demonstrates robust performance under moderate self-occlusion and spherical distortion. While tested in a constrained setting, its modular, real-time design makes it a promising candidate for human–robot interaction and other motion analysis applications, enabling minimal, markerless, and anatomically interpretable upper-limb tracking from omnidirectional vision.

External IDs:dblp:conf/sii/DHaeneCYW26