Can Virtual Agents Scale Up Mentoring?: Insights from College Students' Experiences Using the CareerFair.ai Platform at an American Hispanic-Serving Institution

Yuko Okado, Benjamin D. Nye, Angelica Aguirre, William R. Swartout

Abstract: Mentoring promotes underserved students’ persistence in STEM but is difficult to scale up. Conversational virtual agents can help address this problem by conveying a mentor’s experiences to larger audiences. The present study examined college students’ \((N = 138)\) utilization of CareerFair.ai, an online platform featuring virtual agent-mentors that were self-recorded by sixteen real-life mentors and built using principles from the earlier MentorPal framework. Participants completed a single-session study which included 30 min of active interaction with CareerFair.ai, sandwiched between pre-test and post-test surveys. Students’ user experience and learning gains were examined, both for the overall sample and with a lens of diversity and equity across different, potentially underserved demographic groups. Findings included positive pre/post changes in intent to pursue STEM coursework and high user acceptance ratings (e.g., expected benefit, ease of use), with under-represented minority (URM) students giving significantly higher ratings on average than non-URM students. Self-reported learning gains of interest, actual content viewed on the CareerFair.ai platform, and actual learning gains were associated with one another, suggesting that the platform may be a useful resource in meeting a wide range of career exploration needs. Overall, the CareerFair.ai platform shows promise in scaling up aspects of mentoring to serve the needs of diverse groups of college students.