KeyFlow: Acoustic Motion Sensing for Cursor Control on Any Keyboard

Yujia Liu, Qihang Shan, Zhihao Yao, Qiuyu Lu

Abstract: Despite typing being a critical operation in the digital age, users still need to frequently switch between the mouse and keyboard while typing. We introduce KeyFlow, a tool that integrates mouse functionality into the keyboard through machine learning, allowing users to glide their fingers across the keyboard surface to move the cursor. The whole process does not press the keys down to differentiate from normal typing and avoid false touches. KeyFlow uses any computer-built-in microphones to capture the acoustic features of these gliding gestures, requiring no specialized equipment and can be set up and tested independently within 5 minutes. Our user research indicates that, compared to traditional keyboard and mouse methods, this system reduces hand movement distance by 78.3%, making the typing experience more focused.

External IDs:doi:10.1145/3672539.3686348