In Situ Visual Feedback for Learning Hand Gestures in VR

Anonymous

Keywords: gestural interaction, virtual reality, continuous visual feedback, hand tracking

TL;DR: Design and experimental evaluation of visual feedback for learning hand gestures

Abstract: We investigate In Situ visual feedback, on or in the hand, to learn hand gestures more precisely in virtual reality environments. This feedback helps users understand if any of their fingers are incorrectly positioned during a gesture, and enables hitting a smaller, more precise "target" in pose space. In Situ feedback can be used to teach gestures in a tutorial mode, or can appear automatically when the user gets close enough to a known hand pose, serving both as an autocompletion hint (i.e., if the user terminates the gesture, they know with confidence how it will be interpreted) and as a feedforward hint (i.e., the user can adjust their pose to be closer to the ideal before terminating, when precision is important). We present four variants of In Situ feedback, which were compared in a first study in virtual reality involving 12 users wearing a Meta Quest 2 headset. The most promising variant, Puppet, was evaluated in a second study with 20 users, and enabled greater precision than a static grid, and was preferred by most users.

Track: HCI/visualization