Parsing Post-deployment Students' Feedback: Towards a Student-Centered Intelligent Monitoring System to Support Self-regulated Learning

Prachee Alpeshkumar Javiya, Andrea Kleinsmith, Lujie Karen Chen, John Fritz

Abstract: A student-centered intelligent monitoring system collects data from students and provides insights and feedback to students about various aspects of the learning process. It often leverages data collected from educational technology systems, such as Learning Management Systems (LMS), to support students’ self-regulated learning. A well-designed system needs to strike a delicate balance between the power of machine intelligence (e.g., automatic characterization and inference of students’ behaviors) and the need to promote students’ agency (e.g., the desire to be responsible and take control of their own behaviors). This paper presents a comprehensive qualitative analysis of anonymous student survey data collected from over 500 students with experience with a Learning Activity Monitoring System (LAMS), which has been in operation for over a decade in a public minority-serving higher education institute in the US. The study offers valuable insights into the effectiveness and user perceptions of LAMS they use in their learning context. This analysis reveals the sense-making process or lack thereof, and its implication in exploiting the potential utility of the LAMS. The findings also highlight students’ varied expectations and requirements, providing critical insights for the ongoing development and refinement of the LAMS system toward an intelligent monitoring system that truly centers students’ agency and promotes self-regulated learning. This study contributes to the growing body of work in hearing and understanding students’ genuine voices and the sparse literature of large-scale qualitative analysis of students’ feedback during the post-deployment phase on student-facing data-driven monitoring systems in an ecologically valid context in higher education.