Wikipedia in the Era of LLMs: Evolution and Risks

TMLR Paper6455 Authors

Abstract: In this paper, we present a comprehensive analysis and monitoring framework for the impact of Large Language Models (LLMs) on Wikipedia, examining the evolution of Wikipedia through existing data and using simulations to explore potential risks. We begin by analyzing article content and page views to study the recent changes in Wikipedia and assess the impact of LLMs. Subsequently, we evaluate how LLMs affect various Natural Language Processing (NLP) tasks related to Wikipedia, including machine translation and retrieval-augmented generation (RAG). Our findings and simulation results reveal that Wikipedia articles have been affected by LLMs, with an impact of approximately 1% in certain categories. If the machine translation benchmark based on Wikipedia is influenced by LLMs, the scores of the models may become inflated, and the comparative results among models could shift. Moreover, the effectiveness of RAG might decrease if the knowledge has been contaminated by LLMs. While LLMs have not yet fully changed Wikipedia's language and knowledge structures, we believe that our empirical findings signal the need for careful consideration of potential future risks in NLP research.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

1. **Extended time coverage:** We now include data spanning from 2018 to 2025\.
2. **Direct Impact 1 – Word Frequency:** We have provided a more detailed explanation of word combination selection in the main text for better clarity.
3. **Direct Impact 3 – Page Views:** To enhance generalizability beyond English Wikipedia, we now include page view analysis of Featured Articles in four major language editions: German (de), Spanish (es), English (en), and French (fr). And our conclusion is still consistent.
4. **Indirect Impact 1 – Machine Translation:** We have revised the benchmark construction description for our machine translation experiments to improve clarity.
5. **Indirect Impact 2 – RAG:** The Questioning Methods section has been updated to explicitly reference the prompts used for each method and to clarify the meaning of the setting names shown in Figure 8\. We further conduct experiments using Gemini3-revised Wikinews.

Assigned Action Editor: ~Candace_Ross1

Submission Number: 6455