Beyond Motion Patterns: An Empirical Study of Physical Forces for Human Motion Understanding

Anh Dao, Manh Tran, Yufei Zhang, Xiaoming Liu, Zijun Cui

Keywords: Human Motion Understanding, Gait Recognition, Physics

TL;DR: We show that estimated physical forces are a practical complementary modality for human motion understanding, yielding gains across recognition and captioning tasks.

Abstract:

Human motion understanding has advanced rapidly through vision-based progress in recognition, tracking, and captioning. Yet most existing methods rely on appearance and kinematics, overlooking physical cues such as joint actuation forces that are fundamental in biomechanics. In this work, we revisit motion understanding from a physics perspective and ask a focused question: do physically inferred forces provide complementary information, and under what conditions? To answer this, we augment established baselines with inferred force signals and evaluate their effects across three major tasks: gait recognition, action recognition, and fine-grained video captioning. Across 8 benchmarks, incorporating forces yields consistent performance gains; for example, on CASIA-B, Rank-1 gait recognition accuracy improved from 89.52\% to 90.39\% (+0.87), with larger gain observed under challenging conditions: +2.7\% when wearing a coat and +3.0\% at the side view.
In action recognition, CTR-GCN achieved +2.00\% on Penn Action, while high-exertion classes like punching/slapping improved by +6.96\%. Our findings suggest that force cues encode complementary information beyond visual and kinematic features, establishing a clear empirical foundation for future research on incorporating physical forces for human motion understanding.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 3