EmoGrowth: Incremental Multi-label Emotion Decoding with Augmented Emotional Relation Graph

Kaicheng Fu, Changde Du, Jie Peng, Kunpeng Wang, Shuangchen Zhao, Xiaoyu Chen, Huiguang He

Abstract: Emotion recognition systems face significant challenges in real-world applications, where novel emotion categories continually emerge and multiple emotions often co-occur. This paper introduces multi-label fine-grained class incremental emotion decoding, which aims to develop models capable of incrementally learning new emotion categories while maintaining the ability to recognize multiple concurrent emotions. We propose an Augmented Emotional Semantics Learning (AESL) framework to address two critical challenges: past- and future-missing partial label problems. AESL incorporates an augmented Emotional Relation Graph (ERG) for reliable soft label generation and affective dimension-based knowledge distillation for future-aware feature learning. We evaluate our approach on three datasets spanning brain activity and multimedia domains, demonstrating its effectiveness in decoding up to 28 fine-grained emotion categories. Results show that AESL significantly outperforms existing methods while effectively mitigating catastrophic forgetting. Our code is available at https://github.com/ChangdeDu/EmoGrowth.

Lay Summary:

How can AI systems continuously learn new emotions while recognizing multiple emotions at the same time? Traditional emotion recognition models struggle with this because they either forget past emotions or fail to adapt to new ones.

Our work introduces AESL, a method that solves this by (1) using an "Emotion Relation Graph" to connect different emotions and fill in missing labels, and (2) incorporating psychological knowledge to help the system prepare for future emotions. Surprisingly, our approach not only handles 28+ fine-grained emotions but also avoids forgetting old ones—something previous methods couldn’t achieve.

This breakthrough means AI can better understand complex, real-world emotions, improving applications like mental health support and human-computer interaction. It also challenges the assumption that emotion recognition must trade off flexibility for stability.

Primary Area: Applications->Health / Medicine

Keywords: Emotion decoding， Class incremental learning， Multi-label learning

Submission Number: 2533