Automatic Scoring of Arabic Essays: A Parameter-Efficient Approach for Grammatical Assessment

Somaia Mahmoud, Emad Nabil, Marwan Torki

Abstract: Automatic essay scoring (AES) utilizes computational methods to evaluate written essays, offering an efficient approach to grading in various educational contexts. Essays can have multiple scoring criteria. Instead of building separate systems for each criterion, it’s better to have a single system capable of scoring multiple criteria. This paper introduces a cost-effective and efficient approach to Arabic automated essay scoring, focusing on the evaluation of grammaticality. The proposed approach can also be used to score essays based on other criteria such as similarity to model answer, organization, and adherence to prompt. The approach leverages the pre-trained AraBART model, employing different parameter-efficient methods. We start with Parameter-Efficient Fine-Tuning (PEFT) which optimizes a minimal set of parameters for each specific criterion. Then we apply additional parameter-efficient strategies such as Model Soup, Multi-Round Inference, and Edit Merging. To validate the effectiveness of our approach, we conduct experiments on multiple datasets. The primary objective is to enhance the grammatical correctness of texts and subsequently assess them based on the number of errors. We evaluate the approach on QALB-2014, QALB-2015, and ZAEBUC datasets. The results show similar and sometimes better performance than full fine-tuning while reducing computational and hosting costs. This efficiency is attributed to the utilization of parameter-efficient strategies that entail minimal additional parameters. By scoring essays based on multiple criteria, our approach offers a versatile and flexible tool for educators and language professionals, facilitating efficient evaluation of Arabic essays.