Facial Expression Recognition in the Wild from Attention to Vision Transformer based CNNs

Kuan-Hsien Liu, Xiang-Kun Shih, Tsung-Jung Liu

Abstract: Facial expression recognition in the wild could be affected by irregular occlusions, inconsistent face angles and varying light levels on facial images. In this work, we proposed a model containing facial attention module, residual in residual dense block and vision transformer to tackle with this problem. The facial attention module can assist the model to extract more useful features. Residual in residual dense block and vision transformer can help the model extract more detailed features. In addition to, the number of images of different facial expressions in the dataset is also very different. We proposed a new loss function to deal with this issue. Experimental results on two in-the-wild datasets show that our model is indeed effective. We also created a new facial expression dataset called Fairfaceplus, which is an extension of FairFace, with annotation of expression categories to the original labels. The code of our proposed method will be available on GitHub https://github.com/dreampledge/fairfaceplus.