Hear Here: Sonification as a Design Strategy for Robot Teleoperation Using Virtual Reality

Joe Simmons, Alison Bown, Paul Bremner, Verity McIntosh, Tom Mitchell

Keywords: sonification, sound design, virtual reality, robotics, teleoperation, nuclear decommissioning

TL;DR: This paper explores a novel approach to sonification design for teleoperated robotics in VR in the context of nuclear decommissioning.

Abstract: This paper introduces a novel methodology for the sonification of data, and shares the results of a usability study, putting the methodology into practice within an industrial use case. Working with partners at Sellafield nuclear facility, we explore the effectiveness of the sonification strategy in a scenario where operators in a virtual reality simulation manage a team of semi-autonomous robots. The robots use onboard sensors to map an unexplored space within a simulated nuclear facility, and the sensor data is sonified, using principles of cognitively aligned metaphor, legibility and user comfort. Operators use the auditory information in the VR scene to identify key hazards: radiation, temperature and flammable gas. They are tasked with tagging contaminated areas and protecting the robots from harmful exposure to hazards. We offer insight into features that impact user experience and system usability within our scenario. In particular we examine the importance of aligning AI robot behaviours with user expectations, and the implications of alarm fatigue when high priority sounds are repeatedly triggered. We also discuss the potential limitations of the use case, including the suitability of virtual reality environments of this nature for regular, durational use in the workplace. Overall, results of the study suggest that our distinctive approach to sonification as a design strategy for the teleoperation of robots using virtual reality is effective and easily adopted.

Submission Number: 7