University Student Surveys Using Chatbots: Artificial Intelligence Conversational Agents

Noorhan Abbas, Thomas Pickard, Eric Atwell, S. Aisha Walker

Abstract: Predefined web surveys are often used to collect course evaluations from students in higher education institutions. These institutions use the evaluations to adjust their courses’ pedagogical standards and lecture style to cope with an increasingly uncertain and complex world. Many limitations to using web surveys have been reported such as low response rates and low-quality responses to open questions. To overcome these limitations, artificial intelligence conversational agents (CAs) or ‘chatbots’ are used to play the interviewer role, facilitating the enhancement of the quality of responses. This is accomplished by mimicking human-human conversations; by asking questions in a friendly, casual way and pursuing high-quality responses. This study aims to explore the opportunities and the obstacles of using CAs in collecting course evaluations in three European universities (UK, Spain and Croatia) and one Centre of excellence in Cyprus. The transcripts collected have been analyzed using statistical data analysis methods and qualitative data analysis techniques. Our findings reveal that the use of CAs in collecting course feedback from students has a positive impact on response quality and can boost students’ enjoyment levels. Furthermore, gender differences and student age have been identified as important factors that can influence the depth of the conversation with the CA.