Attach That There: Investigating 3D Virtual Assembly Assistants That Point Into the Real World

Anonymous

Keywords: Embodied conversational agent, pointing, virtual reality

TL;DR: We present a virtual assistant in a spherical FTVR display with the ability to to explain assembly steps and point to indicate real-world targets.

Abstract: Gestures are a fundamental part of human communication. However, commonly used voice assistants do not exploit the advantages of human-like nonverbal communication. We present an Embodied Conversational Agent (ECA) with the ability to explain assembly steps and point to indicate real-world targets. To enable accurate pointing into the real world, we implemented our ECA in a spherical Fish Tank Virtual Reality (FTVR) display. We evaluated the effect of a pointing ECA on the performance and experience in an assembly scenario, as well as investigated whether spherical FTVR displays provide an advantage over 2-dimensional (2D) flat displays. Results show that, while the spherical FTVR was preferred in all conditions, pointing to real pieces did not reduce assembly time or errors compared to showing virtual pieces by holding them up. Based on our findings, we provide design insights and research directions for ECAs with pointing gestures in an assembly scenario.

Track: HCI/visualization

Revision: No