Modeling Score Estimation for Japanese Essays with Generative Pre-trained Transformers

Anonymous

Abstract: This paper describes Japanese essay grading models with Generative Pre-trained Transformers (GPTs) in Japanese. Previous studies of essay grading show that neural network based models utilizing pre-trained language modelｓ such as BERT are effective for several essay data. With the recent rapid development of downloadable GPTs, which are trained on significantly larger datasets compared to BERT, it has become feasible to employ GPTs for the task of essay grading through fine-tuning with Low-Rank Adaptation (LoRA). Most models in previous studies have been applied to English essay data and evaluated for the accuracy, but it is not clear how much prediction accuracy can be achieved for Japanese essays, where linguistic resources are limited. Thus, we apply several Japanese GPTs into Japanese essay data with 12 prompt composed of 4 themes. The experimental results show that a model pre-trained from the beginning with Japanese data has higher accuracy than a model additionally pre-trained from multilingual Llama.

Paper Type: long

Research Area: NLP Applications

Contribution Types: Approaches to low-resource settings, Publicly available software and/or pre-trained models

Languages Studied: Japanese