A New Visualization Framework for Multimodal Programming Attention in Debugging Test

Jiayi Zeng, Wei Liu, Ying Zhou, Xingyu Li, Wenqing Cheng, Qi Ruan

Abstract: The programmer’s attention have been utilized for the behavioral research in programming education. Existing approaches mainly focused on the programmer’s visual attention measured by gaze trackers. However, for the scenarios of programming and debugging, the measurement on just eye movements is not sufficient for understanding programming attention. The measurements on programmer’s other modals of behaviors, including mouse clicks and keyboard inputs, are also required. In this paper, we focus on the programming attention visualization problem in the debugging test scenario. A series of attention visualization approaches are proposed for the two main types of interactions, i.e. reading source code and editing/debugging by keyboard and mouse. We develop methods to visualize student’s attentions from either reading or debugging interactions, and then merge them into multimodal programming attention. The experiment results show that, the proposed multimodal attention graphs can show the diversity of students’ debugging strategies in terms of reading and editing focus in code line and time dimensions. It can help the teacher infer the students’ behavioral intentions and conduct individual diagnose in debugging tests.