Reliability-aware label distribution learning with attention-rectified for facial expression recognition

Liyuan Peng, Yanbing Liu, Yuyun Wei, Jia Cui, Meng Qi

Abstract: Facial expression recognition poses a significant challenge in computer vision with numerous applications. However, existing FER methods need more generalization ability and better robustness when dealing with complex datasets with noisy labels. We propose a label distribution learning model, RA-ARNet, with novel reliability-aware (RA) and attention-rectified (AR) modules to handle noisy labels. Specifically, the RA module evaluates the reliability of the image’ neighboring instances in the valence-arousal space and constructs corresponding label distribution based on the evaluation as auxiliary supervision information to enhance the model’s robustness and generalization on various FER datasets with noisy labels. The AR module can gradually improve the model’s ability to extract attention features of facial landmarks by introducing consistency detection of attention feature maps of images and landmarks in training, thereby improving the model’s FER accuracy. The competitive experimental results on public datasets validate the effectiveness of the proposed method and compare it with the current state-of-the-art methods. The experimental results indicate that the classification performance of RA-ARNet reaches 91.36% on RAF-DB and 61.47% on AffectNet (8 cls) and shows potential to deal with images with occlusion.