Towards Understanding Fixes of SonarQube Static Analysis Violations: A Large-Scale Empirical Study

Ping Yu, Yijian Wu, Jiahan Peng, Jian Zhang, Peicheng Xie

Abstract: Automated static analysis tools (ASATs) have become an integrated part of the software development workflow in many projects. While developers benefit from these tools to deliver quality code conforming to the pre-defined static analysis rules, it has been reported that many ASATs are underused. A number of detected violations are overlooked by developers due to false alarms or unactionable alerts. Despite of existing studies on the fixes of static analysis violations, there is still a gap in collecting and understanding the fact that some types of violations are fixed more often and/or more quickly than other types. To fill this gap, we conduct a large-scale empirical study on 56,506,892 violations from 30 active, popular, and high-quality open-source Java projects with long evolution histories. All violations were traced between adjacent revisions before we filtrated the fixed violations out of the closed ones by considering the types of source code changes that closed the violations. We identified the violation types with the highest and lowest fix rates and those that were fixed the most timely and least timely, and further investigated the possible underlying reasons for the differences in fix rate and fix time. Our findings is helpful to characterize and understand developers’ considerations when fixing violations and provide practical implications for developers, tool builders and researchers to optimize the usage and design of ASATs.