Hand-free natural user interface for VR HMD with IR based facial gesture tracking sensor

Jinhyuk Kim, Jaekwang Cha, Hojun Lee, Shiho Kim

Abstract: We proposed a hand-free Natural User Interface (NUI) for VR Head-Mounted-Display (HMD) with infra-red (IR) based sensor tracking facial gestures. We have realized NUI based on the real-time recognition of user intuitions for VR HMD without any additional control devices except for a built-in Gyroscope and IR couplers with readout circuitry integrated in the foam interface of an HMD. We implemented seven control commands affordable for 3D interactions with virtual objects. The experimental data show that the proposed system provides a convenient and efficient 2D/3D user interface for both manipulating objects and controlling commands while wearing a VR HMD.