Synthesizing Software Engineering Data in a Test-Driven Manner

Lei Zhang, Jiaxi Yang, Min Yang, Jian Yang, Mouxiang Chen, Jiajun Zhang, Zeyu Cui, Binyuan Hui, Junyang Lin

Abstract:

We introduce **SWE-Flow**, a novel data synthesis framework grounded in Test-Driven Development (TDD).
Unlike existing software engineering data that rely on human-submitted issues, **SWE-Flow** automatically infers incremental development steps directly from unit tests, which inherently encapsulate high-level requirements.
The core of **SWE-Flow** is the construction of a Runtime Dependency Graph (RDG), which precisely captures function interactions, enabling the generation of a structured, step-by-step *development schedule*.
At each step, **SWE-Flow** produces a partial codebase, the corresponding unit tests, and the necessary code modifications, resulting in fully verifiable TDD tasks.
With this approach, we generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, creating the **SWE-Flow-Eval** benchmark.
Our experiments show that fine-tuning open model on this dataset significantly improves performance in TDD-based coding.
To facilitate further research, we release all code, datasets, models, and Docker images at [Github](https://github.com/Hambaobao/SWE-Flow).

Lay Summary: Modern software development often involves writing tests to ensure code works correctly. Inspired by this practice, we created **SWE-Flow**, a new system that helps machines learn how to build software step by step by following test instructions. Unlike previous methods that depend on manually written issue reports, **SWE-Flow** learns directly from unit tests, which naturally reflect what the software is supposed to do. At each step, it provides a small piece of code, a test to check it, and suggestions on what needs to be changed—just like how a real developer would work. We used this approach to build a large dataset from real-world projects and showed that it helps AI models do a better job at writing software in a test-driven way. All our tools, data, and code are freely available at https://github.com/Hambaobao/SWE-Flow.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https://github.com/Hambaobao/SWE-Flow

Primary Area: Applications

Keywords: Large language Model, Software Engineering, Test-Driven Development, Code Agent

Submission Number: 15658