Automatic group affect analysis in images via visual attribute and feature networks

Shreya Ghosh, Abhinav Dhall, Nicu Sebe

Abstract:

This paper proposes a pipeline for automatic group-level affect analysis. A deep neural network-based approach, which
leverages on the facial-expression information, scene information and a high-level facial visual attribute information is
proposed. A capsule network-based architecture is used to
predict the facial expression. Transfer learning is used on
Inception-V3 to extract global image-based features which
contain scene information. Another network is trained for
inferring the facial attributes of the group members. Further, these attributes are pooled at a group-level to train a network for inferring the group-level affect. The facial attribute
prediction network, although is simple yet, is effective and
generates result comparable to the state-of-the-art methods.
Later, model integration is performed from the three channels. The experiments show the effectiveness of the proposed
techniques on three ‘in the wild’ databases: Group Affect
Database, HAPPEI and UCLA-Protest database.