Raising the Bar: Automating Consistent and Equitable Student Support with LLMs

Meenakshi Mittal, Azalea Bailey, Victoria Phelps, Mihran Miroyan, Chancharik Mitra, Rishi Jain, Rose Niousha, Gireeja Ranade, Narges Norouzi

Abstract: Large Language Models (LLMs) can be used to automate many aspects of the educational field. In this paper, we look into the benefits of automating responses to student questions in course discussion forums using our Retrieval-Augmented Generation (RAG)-based LLM pipeline (Edison). Our research questions are:RQ1 How do the responses generated by Edison compare to those of TAs in terms of level of detail and use of examples?RQ2 How does the tone of responses generated by Edison compare to that of TA responses?RQ3 Are responses generated by Edison more self-consistent than TA responses?Our results suggest that Edison generates responses with more detail, examples, positive tone, and self-consistency than TAs. We envision Edison being used as a baseline for TAs to build responses on, reduce response times, and promote equitable feedback.