Towards Semi-Supervised Deep Facial Expression Recognition with An Adaptive Confidence Margin

Hangyu Li, Nannan Wang, Xi Yang, Xiaoyu Wang, Xinbo Gao

Abstract:

Only parts of unlabeled data are selected to train models for most semi-supervised learning methods, whose confidence scores are usually higher than the pre-defined threshold (i.e., the confidence margin). We argue that the recognition performance should be further improved by making
full use of all unlabeled data. In this paper, we learn an
Adaptive Confidence Margin (Ada-CM) to fully leverage all
unlabeled data for semi-supervised deep facial expression
recognition. All unlabeled samples are partitioned into two
subsets by comparing their confidence scores with the adaptively learned confidence margin at each training epoch:
(1) subset I including samples whose confidence scores are
no lower than the margin; (2) subset II including samples
whose confidence scores are lower than the margin. For
samples in subset I, we constrain their predictions to match
pseudo labels. Meanwhile, samples in subset II participate
in the feature-level contrastive objective to learn effective
facial expression features. We extensively evaluate AdaCM on four challenging datasets, showing that our method
achieves state-of-the-art performance, especially surpassing fully-supervised baselines in a semi-supervised manner. Ablation study further proves the effectiveness of our
method. The code will be publicly available.