Evaluating an AI Tutor for Bias Across Different Foundation Models

Aditya Vinodh, Emma Harvey, Husni Almoubayyed, Renzhe Yu, Christopher Brooks, Allison Koenecke, Rene F. Kizilcec

Abstract: AI tutors are increasingly deployed to diverse groups of learners, raising the need to provide high-quality responses independent of the identity of learners who use them. We present a collaborative audit that assesses whether LiveHint AI, a large language model-based AI tutor that is currently under development by Carnegie Learning, meets this goal. We repeatedly prompt LiveHint AI with realistic student queries modified to include explicit or implicit statements of identity; e.g., identifying as a particular nationality or writing in a particular dialect. We then assess the responses based on their tone and level of detail. By evaluating different versions of LiveHint AI powered by GPT-4, GPT-4o, and Claude-3.5-Sonnet, we found that the choice of foundation model impacts the level of differentiation in responses. This differentiation may reflect pedagogical strategies (e.g., reducing text complexity when observing typos) or it may be undesirable (e.g., responding to an English prompt in a different language). Education researchers can use this approach to select foundation models that best fit their pedagogical approach, and build guardrails around potentially biased, inconsistent, or undesired behavior.

External IDs:doi:10.1007/978-3-031-98465-5_43