How Large Language Models Write FakeNews Like Humans Do

ACL ARR 2024 December Submission1805 Authors

Abstract:

Fake news detection is a challenging task in the field of natural language processing. The existing methods for detecting manually written fake news still face significant challenges, primarily due to the lack of fake news datasets that imitate human writing styles.
Manual methods often require significant human and material resources, while automated methods generate fake news that diverges from human writing styles. To address these challenges, we propose a novel framework based on Large Language Model (LLM) for generating human-like fake news datasets.
Specifically, we first use a large language model to generate two styles of fake news that contradict the main points of the real news article.
Subsequently, the large language model selects the better of the two generated fake news sentences based on the specified evaluation criteria and replaces the main sentence of the original news article, thus constructing fake news while maintaining a human-written style.
Our approach effectively addresses the challenges of constructing fake news datasets and ensures closer adherence to human writing styles. Additionally, it provides insights into enhancing the human-like writing capabilities of LLM. We will release the LLMFAKE dataset constructed using this method, which contains approximately 2.8k examples. Our experimental results demonstrate that fake news detectors trained on LLMFAKE outperform previous baseline methods on two human-written fake news datasets.

Paper Type: Long

Research Area: Generation

Research Area Keywords: Large Language Models,Disinformation Generation, Disinformation Detection

Languages Studied: English

Submission Number: 1805