Learning to mimic programmers gaze behavior for program comprehension improvement

Jeanne Barthélemy, Takatomi Kubo, Takeshi D. Itoh, Kiyoka Ikeda, Kazushi Ikeda

Abstract: Gaze behavior of human coders could allow to improve programmer-aiding tools relying on program comprehension algorithms, as gaze reveals which subsets of source code programmers focus on to understand its function. When real gaze data are unavailable, algorithmic solutions for gaze behavior estimation might be used to integrate gaze behavior in a global programmer-aiding tool pipeline and enhance it. The objective of this paper is the implementation and training of an algorithmic solution to generate trajectories describing gaze behavior, and illustrate how it can enhance an automatic program comprehension task. Even though our approach has limits and is improvable, we successfully implemented an automatic gaze behavior generation algorithm with visibly better performances than a random policy, and also demonstrated that a trained gaze model can improve a simple program comprehension task. This implies that automatic program comprehension improvement with artificially generated gaze trajectories could be directly possible in a programmer-aiding software, without the need for eye gaze tracking devices every time the programmer-aiding software is used on a new code snippet. Workflows similar to ours could enhance programmer-aiding tools, but also benefit society more broadly by giving new insights toward building AI with attention mechanisms.