Neural Networks for Modeling Source Code Edits

Rui Zhao, David Bieber, Kevin Swersky, Daniel Tarlow

Abstract: Programming languages are emerging as a challenging and interesting domain for machine learning. A core task, which has received significant attention in recent years, is building generative models of source code. However, to our knowledge, previous generative models have always been framed in terms of generating static snapshots of code. In this work, we instead treat source code as a dynamic object and tackle the problem of modeling the edits that software developers make to source code files. This requires extracting intent from previous edits and leveraging it to generate subsequent edits. We develop several neural networks and use synthetic data to test their ability to learn challenging edit patterns that require strong generalization. We then collect and train our models on a large-scale dataset consisting of millions of fine-grained edits from thousands of Python developers.

Keywords: Neural Networks, Program Synthesis, Source Code Modeling

TL;DR: Neural networks for source code that model changes being made to the code-base rather than static snapshots of code.