Test code refactoring unveiled: where and how does it affect test code quality and effectiveness?

Luana Almeida Martins, Valeria Pontillo, Heitor A. X. Costa, Filomena Ferrucci, Fabio Palomba, Ivan do Carmo Machado

Abstract: Refactoring has been widely investigated in the past in relation to production code quality, yet little is known about how developers apply refactoring to test code. Specifically, there is still a lack of investigation into how developers typically refactor test code and its effects on test code quality and effectiveness. This paper presents an exploratory empirical study aimed to bridge this gap of knowledge by investigating (1) whether test refactoring actually targets test classes affected by quality and effectiveness concerns and (2) the extent to which refactoring contributes to the improvement of test code quality and effectiveness. First, we performed an exploratory mining software repository to collect test refactoring data of open-source Java projects from GitHub. Then, we statistically analyzed them in combination with quality metrics, test smells, and code/mutation coverage indicators. Furthermore, we measured how refactoring operations impact the quality and effectiveness of test code. Our findings indicate that test refactorings primarily address low-quality test code, as evidenced by test smells and quality metrics. At the same time, we did not find a statistically significant relationship between test refactorings and code/mutation coverage metrics. In addition, test refactorings enhance the coupling, cohesion, and size of the test code, albeit sometimes leading to an increase in certain test smells. We conclude our study by emphasizing the significance of incorporating both quality metrics and test smells into refactoring decisions to enhance the overall quality of test code.