DIPS: A Dyadic Impression Prediction System for Group Interaction Videos

Chongyang Bai, Maksim Bolonkin, Viney Regunath, V. S. Subrahmanian

Abstract: We consider the problem of predicting the impression that one subject has of another in a video clip showing a group of interacting people. Our novel Dyadic Impression Prediction System (DIPS) contains two major innovations. First, we develop a novel method to align the facial expressions of subjects pi and pj as well as account for the temporal delay that might be involved in pi reacting to pj’s facial expressions. Second, we propose the concept of a multilayered stochastic network for impression prediction on top of which we build a novel Temporal Delayed Network graph neural network architecture. Our overall DIPS architecture predicts six dependent variables relating to the impression pi has of pj. Our experiments show that DIPS beats eight baselines from the literature, yielding statistically significant improvements of 19.9% to 30.8% in AUC and 12.6% to 47.2% in F1-score. We further conduct ablation studies showing that our novel features contribute to the overall quality of the predictions made by DIPS.