Phone-based CSI Hand Gesture Recognition with Lightweight Image-Classification Model

Ashkan Arabi, Michael Straus, Zijie Tang, Zhengkun Ye, Yan Wang

Abstract: As years pass, smartphones are becoming a larger part of daily lives, causing users to interact with them more than ever. There are moments, however, when it becomes difficult for the user to operate their device directly. Currently, a user can either touch their devices for direct interaction, or use voice commands for simpler tasks. Although these two methods are very capable means of interacting with the devices, they have their limitations. Touching a physical device is not always practical, while voice commands become ineffective in loud environments. A good example would be if the user is washing dishes in a noisy environment, where neither physical control nor voice commands are convenient. Existing systems of smartphone CSI gesture recognition rely on manual feature extraction which could be hard to implement as gestures grow in number and complexity. We study the feasibility of using lightweight image classification models with minimal preprocessing by implementing and testing the performance of such an architecture. We collect data for five gestures from three setups and two phones, on which our system is able to obtain 90.0% accuracy. Additionally, we investigate the impact of different people, distances, and phones on the system's performance.