Go to ACL ARR 2025 February homepageAuthorship Verification Using Cloze Test with Large Language Models
Abstract: Assignment outsourcing, also known as contract cheating, occurs when a student outsources an assessment task or a part of it from a third party. It has been one of the most pressing ethical issues in university education and was further exacerbated by the wide availability of chatbots based on large language models. We propose a method that has the potential to verify the authorship of a document in question by filling in a close test. A close test with 10 items selected by our method can be used as a classifier with 0.992 accuracy and $F_1$ score of 0.986. We also describe a general method for building a cloze-test-based classifier when the probability of authors and non-authors correctly filling in cloze items is known.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: authorship verification, LLM, cloze test
Contribution Types: NLP engineering experiment, Data analysis
Languages Studied: english, czech, slovak
Submission Number: 342
Loading