Can You Trust a Generative AI Teacher? Verifying the Imperfections of Generative AI for Educational Purposes Using a Zero-Shot Level Classification Problem
Keywords: Generative AI, Limitations of AI in Education, Social and ethical implications of AI education
Abstract: Recently, with the advent of GPT-4 and GPT Store, various educational LLMs have been utilized. However, from an ed- ucational and social perspective, we felt the need to carefully examine this phenomenon. One of the most basic factors in education is to identify the level of learners, and for this pur- pose, teachers should be able to determine the hierarchy and level of knowledge. Our research team used the ARC-E and ARC-C data to define a science question and answer level classification problem, and through experiments, we found that the current LLM still has limitations in clearly distin- guishing the hierarchy and level of knowledge. From an ed- ucational perspective, this result strongly suggests that using LLM for educational purposes may make it difficult to pro- vide appropriate education at the learner’s level, which may undermine the credibility of education.
Submission Number: 2
Loading