A Real-Time Emotion-Aware System Based on Wireless Body Area Network for IoMT Applications

Chang Li, Yingchi Mao, Qian Huang, Weiliang Xie, Xiaoming He, Jie Wu

Abstract: The Internet of Medical Things (IoMT) stimulates the development of intelligent medical applications. As mental disorders become a global problem, emotion recognition has received widespread attention, as it can contribute to more comprehensive mental health monitoring and psychological assessment. Physiological signal-based emotion-aware monitoring is a particularly promising application due to its noninvasive and objective data collection. Recently, multimodal emotion recognition has been enhanced with wireless body area network (WBAN) access to IoMT, where wireless medical sensors are interconnected and abundant signals are acquired conveniently. However, how to synthesize these multisource physiological signals to facilitate emotion recognition is a challenging problem due to their heterogeneity and interference. To solve this problem, we propose a real-time differential multimodal transformer (Diff-MT), where the main components are the differential hyperinformation extraction (DHE) module, the multimodal global cross-attention encoder (MGCE), and the difference-augmented feature fusion (DFF). Ultimately, we endow the system with emotional awareness and distribute the state to IoMT devices. Extensive experiments demonstrate that the proposed Diff-MT exhibits superior performance compared to existing methods on the WESAD and DEAP datasets and is appropriate for IoMT-based healthcare.