Rethinking Code Review Workflows with LLM Assistance: An Empirical Study

Fannar Steinn Aðalsteinsson, Björn Borgar Magnússon, Mislav Milicevic, Adam Nirving Davidsson, Chih-Hong Cheng

Abstract: Background: Code reviews are a critical yet timeconsuming aspect of modern software development, increasingly challenged by growing system complexity and the demand for faster delivery. Aims: We examine how large language models (LLMs) can support code reviews by addressing common inefficiencies and contextual gaps. Method: At WirelessCar Sweden $A B$, we conducted an exploratory field study to identify current challenges, followed by a field experiment with two LLM-assisted review prototypes: one providing upfront, AIgenerated reviews and another enabling on-demand interaction. Both used a retrieval-augmented generation pipeline to assemble relevant contextual information. Results: The field study revealed frequent context switching, insufficient contextual information, and concerns around false positives. In practice, developers generally preferred the AI-led approach, especially for large or unfamiliar pull requests, though preferences varied with codebase familiarity and review risk. Conclusions: LLM-assisted reviews can reduce cognitive load and improve comprehension, with hybrid proactive/on-demand designs best balancing efficiency, trust, and reviewer control.

External IDs:dblp:conf/esem/AdalsteinssonMMDC25