Stop Automating Peer Review Without Rigorous Evaluation

Joachim Baumann, Jiaxin Pei, Sanmi Koyejo, Dirk Hovy

Track: Track 2: Socio-Economical and Future Visions

Keywords: post-AGI peer review, large language models, paper laundering, artificial hivemind effect

TL;DR: Humans must remain in the peer review loop.

Abstract: As AI systems increasingly generate scientific knowledge, the human ability to critically evaluate research becomes more important, not less. Yet large language models offer a tempting solution to address the peer review crisis, risking the automation of the very skills scientists will need most. This position paper argues that **today's AI systems should not be used to produce paper reviews**. We ground this position in an empirical comparison of human- versus AI-generated ICLR 2026 reviews and an evaluation of the effect of automated paper rewriting on different AI reviewers. We identify two critical issues: 1) AI reviewers exhibit a *hivemind effect* of excessive agreement within and across papers that reduces perspective diversity. 2) AI review scores are trivially gameable through *paper laundering*: prompting an LLM to rewrite a paper significantly increases scores from AI reviewers through stylistic changes rather than scientific improvements. However, non-gameability and review diversity are *necessary but not sufficient* conditions for automation. We argue that *addressing the peer review crisis requires a science of peer review automation* that keeps human scientific judgment at the center of the process---especially as we enter an era where that judgment will be needed most.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Presenter: ~Joachim_Baumann1

Format: Maybe: the presenting author will attend in person, contingent on other factors that still need to be determined (e.g., visa, funding).

Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding would significantly impact their ability to attend the workshop in person.

Submission Number: 21