Monocular Person Localization under Camera Ego-Motion

Yu Zhan, Hanjing Ye, Hong Zhang

Keywords: Robot Companions; Human-Centered Automation; Surveillance Robotic Systems

TL;DR: We present a real-time, optimization-based method that enables a robot with a single camera to robustly locate a person despite severe ego-motion by simultaneously estimating the person's 3D position and the camera's 2D attitude.

Abstract: Localizing a person from a moving monocular camera is critical for Human-Robot Interaction (HRI). To estimate the 3D human position from a 2D image, existing methods either depend on the geometric assumption of a fixed camera or use a position regression model trained on datasets containing little camera ego-motion. These methods are vulnerable to severe camera ego-motion, resulting in inaccurate person localization. We consider person localization as a part of a pose estimation problem. By representing a human with a four-point model, our method jointly estimates the 2D camera attitude and the person's 3D location through optimization. Evaluations on both public datasets and real robot experiments demonstrate that our method outperforms baselines in person localization accuracy. Our method is further implemented into a person-following system and deployed on an agile quadruped robot.

Submission Number: 27