Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions

Eray Erturk, Fahad Kamran, Salar Abbaspourazad, Sean Jewell, Harsh Sharma, Yujie Li, Sinead Williamson, Nicholas J Foti, Joseph Futoma

TL;DR: This work develops foundation models of behavioral signals from 2.5B hours of wearable data, achieving strong performance on 57 health tasks, excelling in behavior-driven predictions and further improves when combined with sensor data.

Abstract: Wearable devices record physiological and behavioral signals that can improve health predictions. While foundation models are increasingly used for such predictions, they have been primarily applied to low-level sensor data, despite behavioral data often being more informative due to their alignment with physiologically relevant timescales and quantities. We develop foundation models of such behavioral signals using over 2.5B hours of wearable data from 162K individuals, systematically optimizing architectures and tokenization strategies for this unique dataset. Evaluated on 57 health-related tasks, our model shows strong performance across diverse real-world applications including individual-level classification and time-varying health state prediction. The model excels in behavior-driven tasks like sleep prediction, and improves further when combined with representations of raw sensor data. These results underscore the importance of tailoring foundation model design to wearables and demonstrate the potential to enable new health applications.

Lay Summary: This study explores how wearable devices like smartwatches can be used to detect and monitor a wide range of health conditions. We develop a new machine learning model that learns from more processed, behavior-level signals derived from wearables. These higher-level signals, which reflect things like how a person walks and their activity trends, may be easier to interpret than prior approaches that modeled raw sensor data (e.g., from the optical heart rate sensor), and more useful for detecting real-world health events. The model was tested on a variety of tasks, including predicting a person’s age and sex, identifying long-term conditions like diabetes, and spotting short-term health changes like a respiratory infection or injury. The ability to detect short-term changes in health is especially valuable for personalized health, where early detection and continuous monitoring can support timely care and better health decisions. Overall, this study shows that wearable devices, when paired with advanced models, have the potential to move beyond fitness tracking and help monitor everyday health in more meaningful ways.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Primary Area: Applications->Health / Medicine

Keywords: Wearables, Health applications, Foundation models

Submission Number: 12662