Detection of distant eye-contact using spatio-temporal pedestrian skeletons

Ryusei Hata, Daisuke Deguchi, Takatsugu Hirayama, Yasutomo Kawanishi, Hiroshi Murase

Abstract: Eye contact is an important factor in determining whether a pedestrian is aware of a vehicle. Most conventional eye contact detection methods rely on direct eye-gaze estimation based on eye measurements. This approach is difficult with distant pedestrians, especially in road environments. In contrast, the presence or absence of eye contact from a pedestrian can be determined based on information on their whole body and movement, such as their facial orientation, posture, and turn. In this study, we propose a method to detect eye contact from even distant pedestrians whose direction of gaze would be difficult to measure. The method captures the relationship between face and body information and its changes over time using spatio-temporal skeleton graph convolution. In an experiment using in-vehicle camera images, an accuracy of 88.6% was achieved. The results demonstrate the effectiveness of the proposed approach.