AU-EMO Correlation based zero-shot facial expression recognition with graph convolutional network

WanFei, Ruicong Zhi

Abstract:

Current recognition systems overlook unseen classes, and Zero-shot learning (ZSL) and Generalized ZSL (GZSL) are regarded as effective methods especially with the help of graphs. By introducing low-level Action Units (AUs) correlations, we propose GraphNet-FER, an expanding zero-shot Facial Expression Recognition (FER) model based on a constructed AUs-Emotions correlation graph. First, we use correlations of AUs-AUs and AU-Emotions to construct an implicit and explicit knowledge combined graph, with semantic representations of each category as node embeddings (implicit knowledge) and transition probabilities between classes as edge weights (explicit knowledge). Second, class prototypes reflecting intrinsically relations are generated through graph propagation. Third, a scoring function is utilized to measure the similarity between visual features and class prototypes for prediction. We verify our idea on three datasets for both zero-shot FER and generalized zero-shot FER. Experiment results demonstrate the effectiveness, achieving Top-1 accuracy of 63.18\%
(ZSL) and H-value of 44.03\% (GZSL) on the RAF-DB dataset.