Adopt Machine-Human Collaboration Peer-Review through Computational Research Assessment

Lei You, Lele Cao

Additional Submission Instructions: For the camera-ready version, please include the author names and affiliations, funding disclosures, and acknowledgements.

Track: Track 1: Original Research/Position/Education/Attention Track

Keywords: peer review, metascience, research evaluation, reproducibility, evidence-linked assessment, disagreement escalation, benchmarking, red-teaming

TL;DR: This paper proposes a position: Computational Research Assessment should standardize machine-human peer review via disagreement escalation, evidence-linked critiques, and community benchmarks/red-teaming to improve rigor, fairness, and scale.

Abstract: Scientific output is outgrowing human review capacity, while AI is already used to draft papers. Authors scale with machines; reviewers largely do not. This asymmetry turns quality control into a bottleneck and increases the risk of both false rejection of high-novelty work and acceptance of flawed results. We propose Computational Research Assessment (CRA) as a discipline-level, method-agnostic agenda for machine-human collaboration in peer review. CRA rests on three principles: treat disagreement as a signal that triggers escalation instead of averaging; make every critique evidence-linked, reproducible, and contestable; and build a community immune system with open corpora, benchmarks, and red-team tests to surface gaming and bias. We map these principles to a co-review engine, a community commons, and theoretical foundations, and we outline near-term pilots and falsifiable commitments, informed by an emerging production-grade pre-review system deployed in the wild.

Submission Number: 278