TWIZ: The Multimodal Conversational Task Wizard

Rafael Ferreira, Diogo Silva, Diogo Tavares, Frederico Vicente, Mariana Bonito, Gustavo Gonçalves, Rui Margarido, Paula Figueiredo, Helder Rodrigues, David Semedo, João Magalhães

Abstract: This paper introduces TWIZ, a multimodal conversational task wizard that supports an engaging experience, where users are guided through a multimodal conversation, towards the successful completion of recipes and DIY tasks. TWIZ leverages task guides from WikiHow and Recipe sources, as well as dialog AI-based methods to deliver a rich, compelling, and engaging experience when guiding users through complex manual tasks. TWIZ participated in the Amazon Alexa Prize Taskbot 2021.