Go to IEEE EMBS BSN 2025 Conference homepageA Multimodal AI-Enabled Framework for Characterizing Overeating Behaviors and Consumption Patterns
Confirmation: I have read and agree with the IEEE BSN 2025 conference submission's policy on behalf of myself and my co-authors.
Keywords: Overeating phenotypes, multimodal sensing, supervised contrastive learning, wearable camera
TL;DR: This paper uses multimodal passive sensing and EMA data with supervised contrastive learning to uncover distinct overeating phenotypes.
Abstract: Overeating is a key contributor to obesity, yet identifying and characterizing its underlying causes remains challenging. While prior research has leveraged Ecological Momentary Assessment (EMA) to capture psychological and contextual factors in real-time, few studies have integrated EMA with passive sensing to uncover fine-grained, individualized consumption behaviors. In this work, we present a multimodal framework combining psychological and contextual data from a custom-built EMA app with validated camera-derived meal microstructure features from a neck-worn activity-oriented wearable camera. Across 41 participants, the camera captured 6,343 hours of footage over 312 days, yielding annotated bites, chews, meal start/end times, and dietitian-confirmed caloric intake. Using supervised contrastive learning, we generated meal-level representations, projected them using UMAP, and applied k-means clustering to identify behavioral phenotypes.
We then conducted a z-score analysis to highlight features most distinctive to each cluster. Among the eight discovered groups, three consistently showed high purity for overeating meals (average purity = 0.99), revealing nuanced, data-driven overeating phenotypes that may inform targeted intervention strategies.
Track: 1. Digital Health Solutions (i.e. sensors and algorithms) for diagnosis, progress, and self-management
NominateReviewer: Josiah Hester: josiah@gatech.edu
Shirlene Wang: shirlene@northwestern.edu
Submission Number: 137
Loading