A Deep Learning Approach to Detecting Engagement of Online Learners

M. Ali Akber Dewan, Fuhua Oscar Lin, Dunwei Wen, Mahbub Murshed, Zia Uddin

Abstract: Online learning environments enable learning for the online learners. The motivational factors, like engagement, play an important role in effective learning. However, the learning designers did not take into consideration the motivational factors involved in the learning process. We believe that the next generation of online learning environments should have the functionality of tracking learner's engagement and thus provide personalized interventions. In this paper, we propose a deep learning-based approach to detecting online learners' engagement through using their facial expressions. Two-level (not-engaged and engaged) and three-level (not-engaged, normally-engaged and very-engaged) decisions are made on engagement detection during classification. We use Local Directional Pattern (LDP) to extract person-independent edge features for the different facial expressions and Kernel Principal Component Analysis (KPCA) to capture the nonlinear correlations among the extracted features. The experiment results show that the proposed method achieves a high accuracy in classification of different engagement levels that the learners may show during their online learning activities (e.g., reading, writing, watching video tutorials, and participating in online meetings). The experiments conducted on the Dataset for Affective States in E-Environments (DAiSEE) demonstrate the effectiveness of the proposed method, where the two-level engagement detection achieves a higher accuracy (90.89%) than the three-level engagement detection (87.25%).