Multi-talker Verbal Interaction for Humanoid Robots

Bartlomiej Klin, Ryszard Beniak, Michal Podpora, Arkadiusz Gardecki, Joanna Rut

Abstract: Working in multi-talker mode is viable under certain conditions, such as the fusion of audio and video stimuli along with smart adaptive beamforming of received audio signals. In this article, the authors verify part of the researched novel framework, which focuses on adapting to dynamic interlocutor’s location changes in the engagement zone of humanoid robots during the multi-talker conversation. After evaluating the framework, the authors confirm the necessity of a complementary and independent method of increasing the interlocutor’s signal isolation accuracy. It is necessary when video analysis performance plummets. The authors described the leading cause as insufficient performance during dynamic conversations. The video analysis cannot derive a new configuration when the interlocutor’s speech apparatus moves beyond the expected margin and the video frame rate drops.