ChatGPT vs Sarcasm - How Well do State of the Art Language Models Handle Sarcasm?

Alexandru-Costin Baroiu, Stefan Trausan-Matu

Abstract: The work presented in this paper highlights the ability of state-of-the-art Large Language Models to generate sarcasm. In the recent months, ChatGPT has become increasingly popular and has started a new wave of interest in AI, especially on the non-scientific community. The model has displayed incredible feats, capable of producing human level results in a variety of tasks. To this end, sarcasm generation comes as a natural evaluation of the model's performance, as it was proven to be a highly difficult task, even for humans. As such, ChatGPT was tested in three experimental setups, no instructions, instructed to be sarcastic and instructed to impersonate a sarcastic character. The results generated by the model were benchmarked with a previous version, GPT2. Quantitative and qualitative analyses of the results were conducted, with ChatGPT achieving the highest BLEU score and generating the best results, outclassing its predecessor.