Evaluating ReLink for Traceability Link Recovery in Practice

Ayberk Yasa, Cemhan Kaan Özaltan, Görkem Ayten, Fatih Kaplama, Ömercan Devran, Baykal Mehmet Uçar, Eray Tüzün

Abstract: Traceability is important in the software development life cycle for managing the connections among various software artifacts, particularly pull requests (PRs) and issues. Developers often neglect to link these manually, reducing traceability. Existing algorithms to recover these links have limited application in closed-source projects. In this study, we share the experience of using ReLink, a predictive PR-issue linking tool with visualization capabilities, in a closed-source project environment. ReLink stands out due to its availability as a fully functional web application and its semi-automated nature, which enhances usability. ReLink determines missing links between PRs and issues based on a confidence score normalized between 0 and 100, calculated using text similarity and heuristic rules. The tool's effectiveness was evaluated in both an open-source project and an industry-based case study. In the open-source project, ReLink achieved a top-5 accuracy of 0.80 and a precision of 0.77. In the industrial case study, ReLink's effectiveness was validated by practitioners selecting the correct link from five issue suggestions, resulting in a top-5 accuracy of 0.86 and a mean reciprocal rank (MRR) of 0.84. Through this experience, we offer insights into both the benefits and challenges of implementing traceability link recovery in the industry and provide recommendations for practitioners seeking to bridge traceability gaps efficiently.