Leveraging Past Assignments to Determine If Students Are Using ChatGPT for Their Essays

Yuhui Zhao, Chunhao Zou, Rohit Sridhar, Christopher Cui, Thad Starner

Abstract: The proliferation of powerful large language models with human-like abilities, like ChatGPT, pose serious challenges for educators to enforce academic integrity policies. To address this problem, we propose a novel approach that uses past students' essay submissions dated before the popularization of ChatGPT, and ChatGPT generated essay responses as ground truth to train classifiers to detect ChatGPT usage for current student submissions. Our case study found that, for the same question prompt, student written answers and ChatGPT generated answers are very different. Testing on the ground truth data shows very simple machine learning methods, including multinomial naive Bayes, linear discriminant analysis, and logistic regression, can achieve close to perfect accuracies in detecting ChatGPT generated responses. Using this approach, we suspect around 7% of current student submissions are ChatGPT generated.