Bi-modal regression for Apparent Personality trait Recognition

Nishant Rai

Abstract: The task of the ChaLearn Apparent Personality Analysis: First Impressions Challenge is to rate/quantify personality traits of users in short video sequences. Although the validity of personality judgments from short interactions is questionable, studies show the possibility of predicting attributed traits (First Impressions) using facial [15] and acoustic [13] features. The challenge introduces a newly constructed dataset which consists of manually annotated videos collected from YouTube. In this paper, we present our approach for predicting traits by combining multiple modality specific models. Our models include Deep Networks which focus on leveraging visual information in the given faces, Networks focusing on supplementary information from the background and models using acoustic features. We also discuss another approach for modeling traits as a combination of global and trait-specific variables. We explore methods for extracting fixed length descriptors of videos based on frame-level predictions. We also experiment with various methods for fusing model predictions. We observe that fusion achieves a considerable gain in accuracy over the best stand-alone model, possibly due to utilizing information from all modalities. The proposed method achieves an accuracy gain of approximately 18% above the provided challenge baseline.