When Code Authors Are Agents: A Large-Scale Study of Human–Agent Collaboration in Pull Requests

Anthonia Oluchukwu Njoku, Zohreh Sharafi, Foutse Khomh

Keywords: AI coding agents, Human–AI interaction, Code review dynamics, Collaboration signals

Abstract:

Autonomous coding agents are increasingly participating in collaborative software development by generating repository-level pull requests (PRs) that must be reviewed and integrated by human teams. While prior work has examined the technical characteristics of agent-generated patches, less is known about how autonomous authorship reshapes human collaboration dynamics in real-world workflows. In this paper, we investigate large-scale human–agent collaboration by comparing 40,214 pull requests across 2,807 GitHub repositories, including 33,596 agent-authored PRs from five autonomous coding agents and 6,618 human-authored PRs. We examine differences across three dimensions: integration outcomes, structural characteristics, and collaboration signals.

Our findings reveal a socio-technical trade-off. Agent-authored PRs are integrated significantly faster, yet exhibit lower merge rates overall. Task type moderates this effect: agents outperform humans in documentation tasks but underperform in behavior-changing contributions. Beyond outcomes, collaboration patterns differ systematically. Agent-authored PRs attract proportionally more bot-generated comments and elicit more analytic, less socially oriented review communication. In contrast, human-authored PRs receive more elaborative and socially engaged feedback. Incorporating psycholinguistic features into predictive models significantly improves merge outcome prediction, demonstrating that communication style carries explanatory power beyond structural code characteristics.

These results suggest that autonomous agents do not merely introduce technical artifacts into repositories, but actively reshape review interaction patterns and evaluative behavior. The impact of coding agents is therefore fundamentally socio-technical, highlighting the importance of studying AI systems within authentic human–AI collaborative environments.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public.

Paper Type: Full-length papers (i.e. case studies, theoretical, applied research papers). 8 pages

Reroute: false

Submission Number: 28