Accept or Not? An Empirical Study on Analyzing the Factors that Affect the Outcomes of Modern Code Review?

Dandan Wang, Qing Wang, Junjie Wang, Lin Shi

Abstract: Background: When submitting patches for code review, a patch is expected as the patch that is accepted in the fewest reworks (rounds of revisions) possibly. However, in practice, not all submitted patches can be accepted and integrated into the codebase the first time, since their quality is unsatisfactory or even poor, which might not pass patch review or need multiple revisions. Aims: In order to improve the effectiveness and efficiency of modern code review, this paper aims to identify the factors that impact the the patch acceptance and the rounds of revisions. Method: We first extract 29 factors from four categories: author, patch, static code quality and review. Then, we apply statistical approach to examine which of and how these factors impact the patch acceptance and the rounds of revisions, respectively. Results: Through case studies of 16,950 patches of 18 projects from Android and Eclipse systems, the result shows that: 1) the number of accepted/reviewed patches by an author has a statistically significant effect on the patch acceptance; 2) the kinds of languages and the number of invited reviewers have a statistically significant effect on the rounds of the revisions; 3) static code quality of patches has little impact on both patch acceptance and the rounds of revisions, although complexity has impact on the number of invited reviewers. Besides, there are other factors that have impact on the patch acceptance and rounds of revisions for different systems. Conclusions: Based on the analysis result, this study offers some suggestions for researchers and practitioners to improve the quality of submitted patches and the process of code review.