On the Usability of Structural Similarity for Action Unit Intensity Detection

Gökhan Güney, Maurice Rohr, Sebastian Dill, Christoph Hoog Antink

Abstract: Action Units (AUs) are facial muscle movements defined by the Facial Action Coding System (FACS) which is a standard for facial expression analysis. Manual AU coding is time-consuming and requires expertise. To overcome these problems, studies have made significant progress in the intensity detection of AUs. In this work, we propose a Structural Similarity Index (SSIM)-based regression approach and test it on the BP4D+ dataset over five different AUs, namely AU6, AU10, AU12, AU14, and AU17. Our approach achieved an overall 0.56 Mean Absolute Error (MAE) value and an overall 0.71 Intraclass Correlation (ICC) score, which is comparable to other studies in the literature. These results underlined that the SSIM-based approach can be an effective and promising method for AU intensity detection.