Knowledge Amalgamation for Single-Shot Context-Aware Emotion Recognition

Tristan Cladière, Olivier Alata, Christophe Ducottet, Hubert Konik, Anne-Claire Legrand

Abstract: Fine-grained emotion recognition using the whole context inside images is a challenging task. Usually, the approaches to solve this problem analyze the scene from different aspects, for example people, place, object or interactions, and make a final prediction that takes all this information into account. Despite giving promising results, this requires specialized pre-trained models, and multiple pre-processing steps, which inevitably results in long and complex frameworks. To obtain a more practicable solution that would work in real time scenario with limited resources, we propose a method inspired by the amalgamation process to incorporate specialized knowledge from multiple teachers inside a student composed of a single architecture. Moreover, the student is not only capable of treating all subjects simultaneously by creating emotion maps, but also to detect the subjects in a bottom-up manner. We also compare our approach with the traditional method of fine-tuning pre-trained mod