Automatic Analysis of Pedestrian's Body Language in the Interaction with Autonomous Vehicles

Walter Morales-Alvarez, María José Gómez-Silva, Gerardo Fernández-López, Fernando García Fernandez, Cristina Olaverri-Monreal

Abstract: The work presented on this paper, aims to provide an automated tool to analyze the autonomous vehicle-pedestrian interaction. It detects pedestrians in the environment, by means of the most modern visual detection technology available in the literature, applied to an stereo camera system, and analyses if the driver looks at the vehicle. To achieve this task the algorithm does pose estimation, feature matching, and facial detection to acquire the position of the pedestrians and distinguish if they notice the vehicle. The algorithm calculates the 3D coordinates of a given pedestrian using the vehicle as a reference, tracking the movement of the pedestrian during all the process, thus providing meta-information of this interaction, which allows to process this information at higher levels: e.g. if the pedestrians feels conformable to cross, or if he performs any other maneuver moving away from the trajectory. The proposed algorithm was tested in campus scenarios where pedestrians and vehicle shared the environment, and the results proved the viability, providing a tool, useful for researchers which allows to process high amounts of information without the need of supervision.