Teacher Agency in Implementing Automated Writing Evaluation Systems in a Science Classroom: A Focus Group Study

Yitong Fu, Shamya Karumbaiah, Xunyi Gao, Jais Brohinsky, Dana Gnesdilow, Sadhana Puntambekar

Abstract: Automated Writing Evaluation (AWE) systems have supported students’ learning of science writing and teachers’ evaluation of the writing product. Previous research on AWE has paid limited attention to teachers’ agency in implementing these tools. As key stakeholders, teachers play an important role in the integration of AWE into traditional classrooms. The present study explores teachers’ perceptions of an AWE tool, PyrEval, and their experiences implementing it in their science classroom. We conducted a focus group with five middle-school science teachers who co-designed PyrEval with university researchers. Our thematic analysis reveals teachers’ perceived benefits and limitations of using the AWE tool in their classrooms. Our findings show that teachers perceive the AWE-integrated classrooms as a whole, and adapt their instructional strategies to actualize perceived benefits and minimize perceived constraints. Our study highlights teachers’ agency in using AWE and has implications to the design of instructional support tools.