RefNet: Automatic Essay Scoring by Pairwise Comparison

Jiaxin Li, Jinan Zhou

TL;DR: Automatically score essays on sparse data by comparing new essays with known samples with Referee Network.

Abstract: Automatic Essay Scoring (AES) has been an active research area as it can greatly reduce the workload of teachers and prevents subjectivity bias . Most recent AES solutions apply deep neural network (DNN)-based models with regression, where the neural neural-based encoder learns an essay representation that helps differentiate among the essays and the corresponding essay score is inferred by a regressor. Such DNN approach usually requires a lot of expert-rated essays as training data in order to learn a good essay representation for accurate scoring. However, such data is usually expensive and thus is sparse. Inspired by the observation that human usually scores an essay by comparing it with some references, we propose a Siamese framework called Referee Network (RefNet) which allows the model to compare the quality of two essays by capturing the relative features that can differentiate the essay pair. The proposed framework can be applied as an extension to regression models as it can capture additional relative features on top of internal information. Moreover, it intrinsically augment the data by pairing thus is ideal for handling data sparsity. Experiment shows that our framework can significantly improve the existing regression models and achieve acceptable performance even when the training data is greatly reduced.

Code: https://drive.google.com/open?id=1V2HqjtzTkdTxCQxrjQzEymsheEV2hLSl

Keywords: Natural Language Processing, Automatic Essay Scoring, Few-shot Learning, Neural Network