Learning to Recognize Semantically Similar Program Statements in Introductory Programming Assignments

Mayur Sunil Jawalkar, Hadi Hosseini, Carlos R. Rivero

Abstract: With the continuously increasing population of students enrolling in introductory programming courses, instructors are facing challenges to provide timely and qualitative feedback. Automated systems are appealing to address scalability issues and provide personalized feedback to students. Many of the current approaches fail to handle flexible grading schemes and low-level feedback regarding (a set of) program statements. The combination of program static analysis in the form of program dependence graphs and approximate graph comparisons is promising to address the previous shortcomings. Current techniques require pairwise comparisons of student programs that does not scale in practice. We explore techniques to learn models that are able to recognize whether an unseen program statement belong to a semantically-similar set of program statements. Our initial results on a publicly-available introductory programming assignment indicate that it is possible to assign with high accuracy an individual program statement to some of the popular semantically-similar sets, and a large proportion is covered with these, which suggests feedback provided by instructors can be automatically propagated to other student programs.