When Should an AI Scientist Stop? Verifiable Experiment Steering and Refusal for Autonomous Discovery

Neel Tushar Shah, Manglam Kartik

Additional Submission Instructions: For the camera-ready version, please include the author names and affiliations, funding disclosures, and acknowledgements.

Track: Track 1: Original Research/Position/Education/Attention Track

Keywords: autonomous scientific discovery, AI scientist, Bayesian experimental design, model discrimination, library inadequacy, refuse and revoke, verifiable AI, unresolved subspace, residual guard, governance for autonomous labs

TL;DR: A verification layer that gives AI scientists a first-class "refuse and revoke" signal when the model library is structurally wrong, in the same loop that selects the next experiment.

Abstract:

We present CARTOGRAPH, a verification layer for
AI scientists that couples unresolved-subspace
experiment steering (select), explicit ambiguity
closure (resolve), and residual-based library
inadequacy detection (refuse). Under a local
linear-Gaussian bridge, raw unresolved projection
is the isotropic unresolved Fisher-information
trace, while CARTOGRAPH-A is the exact
unresolved A-optimal rule; closed-form EIG
and Box–Hill arise as local comparators
rather than global equivalents. Across five
testbeds, CARTOGRAPH-A beats raw projection
129W/0T/15L at d = 8 (p < 10−21) in a
replicated structured cascade. More distinctively,
the framework tentatively identifies three out-
of-library pharmacokinetic mechanisms and
then revokes those identifications as residuals
expose structural misfit, while one perturbed
in-library control stays identified throughout. In
low-dimensional pharmacokinetic and filtered
EPA settings, near-ties against disagreement
are predicted by theory and observed. Finally,
in a retrospective audit of 40 positive claims
from the published A-Lab autonomous materials system, the refuse guard flags all 4
claims later marked inconclusive under manual
reanalysis while passing 32/36 confirmed
claims. Codes can be found at the anonymous repository
https://github.com/ai4science-boed/cartograph.git

Submission Number: 64