Exploring Biases in Facial Expression Analysis using Synthetic Faces

Ritik Raina, Miguel Monares, Mingze Xu, Sarah Fabi, Xiaojing Xu, Lehan Li, Will Sumerfield, Jin Gan, Virginia R. de Sa

Keywords: synthetic faces, facial expression recognition, facial action unit estimation, racial bias

TL;DR: Synthetic face images reveal underlying biases among facial expression models.

Abstract: Automated facial expression recognition is useful for many applications, but models are often subject to racial biases. These racial biases may be hard to reveal due to the complexity and opacity of the complex networks needed for state-of-the-art performance. Racial biases are also hard to demonstrate due to the inability to fully match facial expressions across real people. In this paper we use artificially created faces where facial expression can be carefully manipulated and matched across artificial faces with different skin colors and different facial shapes. We show that several public facial expression models appear to have racial biases. This work is an important step towards the eventual goal of understanding the basis of these biases and removing them from facial expression models.