A Student-Role Web-Agent Architecture for Testing Educational Interfaces

Siyang Liu, Yang Jiang, Carolyn Forsyth, Jeremy Y Lee, Rada Mihalcea, Jessica Andrews Todd

Keywords: Web Agent Architecture, AI for Education, Automatic Testing

Abstract:

We present EduWebTester, an architecture for automated testing of educational web interfaces using simulated student-role web agents driven by Vision-Language Models.
The architecture consists of three modules: a Student Simulation Module that generates student behaviors based on configurable profiles, a Web Agent Module that observes interfaces and plans actions, and an Environment Module that manages browser automation and robustly handles dynamic educational pages.
We conduct an empirical study comparing 37 paired simulated and human students on a conversation-based science assessment.
Our findings show that simulated students achieve similar group-level usability results to human students (e.g., both have around 77\% of students supporting a key theme) and identify 8 of 13 of the same specific usability issues.
Simulated students also discover additional bugs and UI design issues that human studies miss.
However, individual-level accuracy remains limited (61\% exact match rate), and simulated students struggle to capture subjective design concerns.
These results suggest that simulated student testing is most effective for usability testing and objective issue detection, while subjective experience simulation requires further research.
Our architecture is versatile for educational web pages, and we will open-source the tool to support the research community.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 53