Towards Social Robots: Automatic Evaluation of Human-robot Interaction by Face Detection and Expression Classification

Gwen Littlewort, Marian Stewart Bartlett, Ian R. Fasel, Joel Chenu, Takayuki Kanda, Hiroshi Ishiguro, Javier R. Movellan

Abstract: Computer animated agents and robots bring a social dimension to hu- man computer interaction and force us to think in new ways about how computers could be used in daily life. Face to face communication is a real-time process operating at a time scale of less than a second. In this paper we present progress on a perceptual primitive to automatically detect frontal faces in the video stream and code them with respect to 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, sur- prise. The face finder employs a cascade of feature detectors trained with boosting techniques [13, 2]. The expression recognizer employs a novel combination of Adaboost and SVM’s. The generalization performance to new subjects for a 7-way forced choice was 93.3% and 97% correct on two publicly available datasets. The outputs of the classifier change smoothly as a function of time, providing a potentially valuable repre- sentation to code facial expression dynamics in a fully automatic and unobtrusive manner. The system was deployed and evaluated for mea- suring spontaneous facial expressions in the field in an application for automatic assessment of human-robot interaction.