K-AirWrite: Real-Time Continuous Air Handwriting Recognition Using Kinect

Zhixiong Yang, Ziyi Zhen, Hui Xu, Xinlong Feng

Abstract: Camera-based air handwriting recognition can enable a new form of user-computer interface. For a practically viable air handwriting recognition system, real-time responsiveness, robustness against environmental and human-induced interferences, and coherent handwriting recognition are essential. However, existing systems fall short of fulfilling these requirements simultaneously. To this end, we propose K-AirWrite, a real-time, robust system designed for continuous air handwriting recognition. The key enabler is to simplify complex video stream recognition into a single-frame trajectory image recognition problem by mapping hand motion in the video stream to hand trajectory coordinate images and enhancing real-time performance through a lightweight detection network. Furthermore, we develop a segmentation method that leverages coordinate changes induced by the inherent pattern during the character-switching process to segment continuous characters. These segmented characters are then sequentially input into the M-RepVGG network for recognition. Extensive experiments demonstrate that K-AirWrite is remarkably efficient and exhibits robust generalization capabilities, achieving an accuracy rate of 98.2% for individual characters and 96.4% for continuous characters.