An Affective Multi-modal Conversational Agent for Non Intrusive Data Collection from Patients with Brain Diseases

Chloe Chira, Evangelos Mathioudis, Christina Michailidou, Pantelis Agathangelou, Georgia Christodoulou, Ioannis Katakis, Efstratios Kontopoulos, Konstantinos Avgerinakis

Abstract: This paper presents Zenon, an affective, multi-modal conversational agent (chatbot) specifically designed for treatment of brain diseases like multiple sclerosis and stroke. Zenon collects information from patients in a non-intrusive way and records user sentiment using two different modalities: text and video. A user-friendly interface is designed to meet users’ needs and achieve an efficient conversation flow. What makes Zenon unique is the support of multiple languages, the combination of two information sources for tracking sentiment, and the deployment of a semantic knowledge graph that ensures machine-interpretable information exchange.