Apparent Personality Recognition from Uncertainty-Aware Facial Emotion Predictions using Conditional Latent Variable Models

Mani Kumar Tellamekala, Timo Giesbrecht, Michel F. Valstar

Abstract: We propose two key ideas to improve the performance of apparent personality traits estimation from face videos: 1. using dimensional emotion predictions fused with face image embeddings as input features and 2. effectively aggregating global temporal context related to personality traits from the input feature sequence. In the former, we propose to use uncertainty-aware predictions of valence and arousal as additional input features along with the face image embeddings. To this end, we first build uncertainty-aware facial emotion recognition models by adopting epistemic (model) and aleatoric (data) uncertainty categorisation framework. In terms of improvement in the personality recognition performance, we show that uncertainty-aware emotion predictions outperform the point estimates of emotions by significant margins. On the other hand, for effectively aggregating the temporal context from the input feature sequence, we propose to use a conditional latent variable model that builds on some recently proposed neural latent variable methods for global context aggregation. By combining these two ideas, our proposed personality recognition method achieves state-of-the-art results on a large-scale in-the-wild dataset, ChaLearn, with ~42 % relative performance improvement over the best of existing benchmarks.