Region-Based Emotion Recognition via Superpixel Feature Pooling

Zhihang Ren, Yifan Wang, Tsung-Wei Ke, Yunhui Guo, Stella X. Yu, David Whitney

Keywords: Emotion recognition, Superpixel, Feature pooling

Abstract: Perceiving other people's emotional states is fundamentally important for successful social interactions. Traditional emotion recognition algorithms exclusively focus on facial expressions, ignoring the critical role of background context, which is now known to be necessary to accurately represent and understand the emotions of others. More recent studies have utilized different fusing techniques to combine facial and contextual information in visual scenes, but these approaches are limited to detection-based methods. In this study, we propose a new region-based emotion recognition pipeline via superpixel feature pooling that does not rely on detection. Our proposed pipeline consists of three types of blocks, including an initial over-segmentation block, the superpixel pooling block, and the emotion recognition block. On EMOTIC and VEATIC datasets, our proposed pipeline improves state-of-the-art performance by 68.57% and 11.79% respectively. We also achieve competitive performance on the CAER-S dataset.

Submission Number: 11