GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos

Garima Sharma, Shreya Ghosh, Abhinav Dhall, Munawar Hayat, Jianfei Cai, Tom Gedeon

Abstract: Estimating the most dominant person in a social interaction setting is a challenging feat even with the advancement of deep learning techniques due to problem complexity, non-availability of labelled data and subjective biases in annotations. This paper aims to reformulate the problem of detecting the Most Dominant Person (MDP) as a person ranking problem by utilizing person-specific attributes such as facial gestures, eye gaze, visual attention and speaking patterns. Our proposed framework, attributed Graph-based dominant person ranking in social InTeracTIon videos, GraphITTI, learns generic and robust person rankings on top of context level features. To inject domain knowledge into the GraphITTI framework, we consider inter-personal and intra-personal aspects along with spatiotemporal context patterns. Our extensive quantitative analysis suggests that GraphITTI framework performs favourably over the current state-of-the-art for dominant person detection and ranking. The code is available at https://github.com/shgnag/GraphITTI.