USee: Ultrasound-Based Device-Free Eye Movement Sensing

Wen Cheng, Mingzhi Pang, Haoran Wan, Shichen Dong, Dongxu Liu, Wei Wang

Abstract: Eye movements play a significant role in human-computer interaction and are widely recognized as an essential health indicator, making their detection both appealing and technically challenging. In this paper, we present a system named USEE that achieves high-precision capture of weak and aperiodic eye movements by utilizing fine-grained and ubiquitous ultrasound signals, capturing both blinking and more subtle saccades. We first identify signal changes associated with eye movements by capturing the unique impact of blinking. Further, we establish a pioneering relationship between the residuals from signal decomposition and subtle eye movements. Utilizing inno-vative signal processing architectures, we mitigate interference and effectively extract eye movement features. Subsequently, we employ one-dimensional convolutional operations in place of signal cross-correlation, designing filters for motion category identification and a lightweight convolutional neural network for saccade direction classification. This enables our system to serve as a foundational sensing layer for eye movement tracking, applicable across diverse applications. We implement USEE on both a research-purpose platform and a commodity Raspberry Pi. Extensive experimental results demonstrate the effectiveness of our system, achieving 91% accuracy in saccade recognition and 94% in blink detection. The system proves robust, even in challenging scenarios with strong interference, such as the presence of moving pedestrians.