Go to DBLP homepageTowards understanding code review practices for infrastructure-as-code: An empirical study on OpenStack projects
Abstract: Infrastructure-as-code (IaC) is a widely used practice to automate the creation, provisioning, orchestration, and configuration of infrastructures (such as networks, databases, and services). IaC allows the use of specification files in the form of code to manage the infrastructure. Practitioners can apply quality assurance best practices like code review to evolve IaC artifacts like any other software system. Code review (CR) is a common practice in which developers (i.e., reviewers) review code changes submitted by their peers to fix errors and ensure that the code adheres to standards. Previous studies have shown that MCR can improve the overall quality of the code, and that MCR practices may vary depending on the code under review. Yet, little is known about how MCR practices are used in the context of IaC compared to Non-IaC code. While Non-IaC focuses on implementing system components, IaC translates these functional requirements into configuration code that defines infrastructure behavior, highlighting the need to explore their distinct review practices. This paper presents the first empirical study to investigate how practitioners perform code reviews for IaC code changes. Using a dataset of over 300k code reviews from the OpenStack ecosystem, we found that both IaC and Non-IaC code changes take a comparable merge time. However, IaC developers make 1.82 times more churn, while reviewers exchange 1.1 times more messages when reviewing IaC-related code changes. We further examined the contribution of experienced reviewers in reviewing IaC code changes. Our findings reveal that the top 5% of reviewers participate in 44% to 75% of IaC code changes, indicating that dedicated reviewers are assigned to review IaC code changes in OpenStack. Finally, to gain a better understanding of the factors influencing reviewers in the evaluation of IaC code changes, we conducted a rigorous thematic analysis on a representative sample of IaC code changes. We created a checklist with seven main topics and 38 sub-topics through a thematic analysis. To validate our checklist, we surveyed nine developers amongst the most active OpenStack reviewers in IaC code changes. This approach strengthens the reliability of our findings and adds empirical support to the identified checklist. This latter can be useful as a foundation of IaC guidelines for developers along the IaC code change development and review process to check the quality of their IaC code changes. In conclusion, we emphasize the importance of recognizing the non-trivial nature of the IaC code change review. We argue for increased attention from researchers to explore other dimensions of IaC code changes, recognizing that these are indispensable practices for ensuring the robustness of software systems infrastructure.
Loading