Go to MSLD 2026 Meeting homepageDebating AI in Education: Public Conversations on Reddit
Keywords: Education, Artificial Intelligence, Social Media, Computational Social Science, Social Norms
TL;DR: We analyze Reddit discussions about AI use in education and use human annotations to evaluate LLMs that can label a larger dataset.
Abstract: Debating AI in Education: Public Conversations on Reddit
The growing use of generative AI tools in educational settings has raised new questions about academic integrity, acceptable use, learning outcomes, and the roles of students and instructors in AI-mediated academic work. Public conversations about these issues are increasingly taking place online, where users openly debate whether using tools such as ChatGPT for studying, writing, feedback, or academic support is helpful, harmful, fair, or dishonest. This project examines how AI use in education is discussed on Reddit, with the goal of understanding how people frame AI-related practices, who they believe is using these tools, and what benefits or risks they associate with them.
Our study uses a dataset of Reddit posts and comments drawn from education-related discussions about AI. To analyze these conversations, separate annotation schemes were developed for posts and comments. For posts, the annotation framework captures whether an AI use case is present, whether the use is actual, hypothetical, multiple, or absent, and the main quoted use case when applicable. It also identifies the scope of AI use, distinguishing between academic logistics and campus life versus learning and assessment activities. Additional post-level categories capture the role of the author, the role of the AI user, whether the author is describing their own use, and several dimensions of the type of use being discussed, including creation, information-related use, advice-related use, motivations for use, frequency of use, and potential risks. These categories allow the analysis to distinguish, for example, whether AI is being used to create an artefact or generate ideas, to analyze or summarize information, to seek guidance or validation, and whether the post raises ethical, accuracy-related, or resource-related concerns.
A separate annotation scheme was designed for comments in order to better capture the evaluative and reactive nature of discussion threads. Comment annotations identify whether an AI use case is present and classify it as actual, hypothetical, multiple, or absent. They also capture the role of the comment author, the role of the person using AI in the described situation, and the overall stance expressed toward the AI use, such as positive, neutral or mixed, negative, or not specified. In addition, the comment framework includes a combined benefits-and-risks category that records whether the comment frames AI in terms of curiosity, efficiency, accessibility, ethical concerns, accuracy and learning risks, resource-related issues, or no explicit consequences.
Together, these post- and comment-level annotation frameworks provide the foundation for both close analysis and model evaluation. The manually annotated data will be used to create test sets for assessing the performance of large language models on the post- and comment-level classification tasks. By comparing model predictions against human annotations, we can evaluate how well LLMs capture categories such as AI use case, social roles, stance, motivations, and perceived risks or benefits. The longer-term goal is to use the validated model to annotate a much larger corpus of Reddit discussions, enabling broader analysis of how AI use in education is framed across online conversations.
Email Sharing: We authorize the sharing of all author emails with Program Chairs.
Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.
Submission Number: 116
Loading