Lean-Interaction: passive image manipulation in concurrent multitasking

Danny Schott, Benjamin Hatscher, Fabian Joeres, Mareike Gabele, Steffi Hußlein, Christian Hansen

Abstract: Complex bi-manual tasks often benefit from supporting visual information and guidance. Controlling the system that provides this information is a secondary task that forces the user to perform concurrent multitasking, which in turn may affect the main task performance. Interactions based on natural behavior are a promising solution to this challenge. We investigated the performance of these interactions in a handsfree image manipulation task during a primary manual task with an upright stance. Essential tasks were extracted from the example of clinical workflow and turned into an abstract simulation to gain general insights into how different interaction techniques impact the user’s performance and workload. The interaction techniques we compared were full-body movements, facial expression, gesture and speech input. We found that leaning as an interaction technique facilitates significantly faster image manipulation at lower subjective workloads than facial expression. Our results pave the way towards efficient, natural, hands-free interaction in a challenging multitasking environment.

Keywords: Multimodal Interaction, Multitasking, Hands-free Interaction, Radiology, Medical Domain