The Verification Bottleneck: Managing Trust in Post-AGI Science

Arul Murugan

Track: Track 1: Technical Foundations for a Post-AGI World

Keywords: verification bottleneck, post-AGI science, epistemic triage, provisional knowledge, scientific trust, AI-generated claims, verification cascades, epistemic pollution

TL;DR: AI now generates discoveries faster than any infrastructure can verify them, creating a verification bottleneck that threatens to flood science with indistinguishable valid and invalid claims unless we adopt new epistemic frameworks.

Abstract:

AI systems can now compress decade-long research programs into days. The
result is a structural asymmetry: discovery scales faster than verification.
Current AI achieves only 21\% recall detecting errors in manuscripts; it
generates plausible claims far better than it verifies them. This
``verification bottleneck'' has been flagged as a concern, but we lack
frameworks for managing it. We contribute three operationalized mechanisms: (1) epistemic triage combining prediction markets, statistical
thresholds, and anomaly detection to prioritize what gets verified; (2)
verification cascades, a hierarchical architecture assigning
epistemic status based on verification depth; and (3) an extension of provisional knowledge to AI-generated claims, with explicit
conditions for status transitions. Drawing on social epistemology and the
sociology of scientific knowledge, we examine how these frameworks address
scalable oversight, trust, and human roles in machine-accelerated science.
Without such frameworks, science risks epistemic pollution: a state where
valid and invalid claims become indistinguishable.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Presenter: ~Arul_Murugan1

Format: Yes, the presenting author will attend in person if this work is accepted to the workshop.

Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding would significantly impact their ability to attend the workshop in person.

Submission Number: 16