Detection of Fake Generated Scientific Abstracts

Panagiotis C. Theocharopoulos, Panagiotis Anagnostou, Anastasia Tsoukala, Spiros V. Georgakopoulos, Sotiris K. Tasoulis, Vassilis P. Plagianakos

Abstract: The widespread adoption of Large Language Models, such as the publicly available ChatGPT has marked a significant turning point in the integration of Artificial Intelligence into people’s everyday lives. The academic community has taken notice of these technological advancements and has expressed concerns regarding the difficulty of discriminating between what is real and what is artificially generated, while researchers already work on developing effective systems to identify machine-generated text. In this study, we examine the performance of different methodological approaches for this task. To achieve this, we utilize the GPT-3 model to generate scientific paper abstracts of real research papers. By conducting this research, we shed light on the capabilities and limitations of current Machine Learning models for discriminating machine-generated text. Simultaneously, we enhance our understanding of the operation of the Large Language Models.