Ghostbuster: Detecting Text Ghostwritten by Large Language Models
Keywords: AI-generated text detection, text generation, large language models
TL;DR: We introduce Ghostbuster, a state-of-the-art system for detecting AI-generated text.
Abstract: We introduce Ghostbuster, a state-of-the-art system for detecting AI-generated text.
Our method works by passing documents through a series of weaker language models and running a structured search over possible combinations of their features, then training a classifier on the selected features to determine if the target document was AI-generated.
Crucially, Ghostbuster does not require access to token probabilities from the target model, making it useful for detecting text generated by black-box models or unknown model versions.
In conjunction with our model, we release three new datasets of human and AI-generated text as detection benchmarks that cover multiple domains (student essays, creative fiction, and news). Ghostbuster averages 99.0 F1 across all three datasets, outperforming previous approaches such as GPTZero and DetectGPT by up to 41.6 F1.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8500
Loading