Contrastive Learning with Auxiliary User Detection for Identifying Activities

Wen Ge, Guanyi Mou, Emmanuel O. Agu, Kyumin Lee

Abstract: Human Activity Recognition (HAR) is an important task in ubiquitous computing, with impactful real-world applications. While recent state-of-the-art HAR research has demonstrated impressive performance, some key aspects remain under-explored. First, we believe that for optimal performance, HAR models should be both Context-Aware (CA) and personalized. However, prior work has predominantly focused on being Context-Aware (CA), largely ignoring being User-Aware (UA). We argue that learning user-specific differences in performing various activities is as critical as considering user context while performing HAR tasks. Secondly, we believe that the predictions of HAR models should be unified, reliably recognizing the same activity even when performed by different users. As such, the representations utilized by CA and UA models should explicitly place different users performing the same activity closer together. Moreover, identifying the user performing an activity is useful in applications such as thwarting cheating by having another person perform medically-prescribed activities. To bridge this gap, we introduce Contrastive Learning with Auxiliary User Detection for Identifying Activities (CLAUDIA), a novel framework designed to address these issues. Specifically, we expand the contextual scope of the CAHAR task by integrating User Identification (UI) within the CAHAR framework, jointly predicting both CAHAR and UI in a new task called User and Context-Aware HAR (UCA-HAR). This approach enriches personalized and contextual understanding by jointly learning user-invariant and user-specific patterns. Inspired by state-of-the-art designs in the visual domain, we introduce a supervised contrastive loss objective on instance-instance pairs to enhance model efficacy and improve learned feature quality. Through theoretical exposition, empirical analysis of real-world datasets, and rigorous experimentation, we demonstrate the significance of each component of CLAUDIA and discuss its relationship with existing methodologies. Evaluation across three real-world CA-HAR datasets reveals substantial performance enhancements, with average improvements ranging from 11.7% to 14.2% in Matthew's Correlation Coefficient (MCC) and 5.4% to 7.3 % in Macro F1 score. To encourage further research, we share code with additional supplement material in repository https://github.com/GMouYes/CLAUDIA.

External IDs:dblp:conf/icmla/GeMAL24