TA Buddy: AI-Assisted Grading Tool for Introductory Programming Assignments

Goda Nagakalyani, Saurav Chaudhary, Varsha Apte, Ganesh Ramakrishnan

Abstract: In introductory programming courses, autograders typically evaluate student programs by running testcases without inspecting the source code. However, educational grading often requires manual code inspection for two key reasons: (1) to award partial marks for code that may fail test cases but is partially correct, and (2) to assign marks based on code quality or specific criteria set by the instructor, such as requiring a particular algorithm, e.g., bubble sort. Rubric-based subjective grading is beneficial for these reasons, but manual grading for large course enrollments is time consuming. This demo introduces TA Buddy, an AI assistant integrated with IIT Bombay's BodhiTree Evalpro platform, which is designed to streamline grading in introductory programming courses. It is powered by a pre-trained code LLM, which was fine-tuned with a dataset created here at IITB Bombay. Its key benefits include speeding up the grading process with AI-generated suggestions for ratings of the criteria of a grading rubric. Furthermore, it provides feedback with justifications for assigned grades, making it useful for large courses where manual grading is time-consuming. Note that TA-Buddy only suggests grades to TAs, TAs are still required to review the grades and accept or reject them. In that sense, TA-Buddy offers an AI-Assisted grading option to TAs. This hybrid approach reduces grading time by up to 45% while maintaining an average match of 90% (on a sample of six problems) with un-assisted manual grades.