Active Grade Estimator on Short Answer Assessment

Mohammad Iqbal, Tsamarah Rana Nugraha, Hsing-Kuo Pao

Abstract: Active learning, as a well-known machine learning strategy, is understood to be advantageous to automated grading systems. Despite teachers’ limited guidance on the systems, we can still expect effective and proper automatic grading results. However, given a high variety of Q/A content, one has to pay attention to a diverse set of correct answers to offer suitable grading. Moreover, students who lack self-confidence in answering exam questions often provide long-winded or formless responses, hindering automated grading systems from producing reliable results. The issues above prohibit applying the conventional keyword-matching approaches to solve the problems. To deal with the difficulty, we propose a novel two-stage query strategy based on large language models (LLMs). In the first stage, we match a student’s answer to a key answer from a prepared answer set. Second, we summarize the selected student’s answer by well-structured sentence(s) via LLMs. The proposed LLM-based approach is considered a prototype-based learning method. On the side, we attempt to reduce the possibility of biased labeling by having multiple graders as oracles to join the active learning framework. The proposed approach has been demonstrated on two public datasets of automated short answer grading with different grading tasks. Overall, the proposed model exhibits outstanding performance when compared to state-of-the-art methods that focus on similar topics in terms of effectiveness and efficiency.

External IDs:dblp:journals/aiedu/IqbalNP25