Multimodal User Interface for Augmented Assembly

Sanni Siltanen, Mika Hakkarainen, Otto Korkalo, Tapio Salonen, Juha Sääski, Charles Woodward, Theofanis Kannetis, Manolis Perakakis, Alexandros Potamianos

Abstract: In this paper, a multimodal system for augmented reality aided assembly work is designed and implemented. The multimodal interface allows for speech and gestural input. The system emulates a simplified assembly task in a factory. A 3D puzzle is used to study how to implement the augmented assembly system to a real setting in a factory. The system is used as a demonstrator and as a test-bed to evaluate different input modalities for augmented assembly setups. Preliminary system evaluation results are presented, the user experience is discussed, and some directions for future work are given.