OpenDiscovery: A Verifiable, Creative Science Problem-Solving Dataset to Forge AI Scientists

Yixuan Weng, QiYao Sun, Minjun Zhu, Yue Zhang

Track: Track 2: Dataset Proposal Competition

Keywords: AI Scientists, Science Problem-Solving, Verifiable

Abstract: The grand vision of an "AI Scientist" is constrained by a fundamental bottleneck: while current models excel at imitating known knowledge, they lack the capacity for autonomous discovery and creative problem-solving. We propose OpenDiscovery, a novel dataset paradigm designed to train and benchmark AI agents for verifiable scientific discovery. Moving beyond static input-output pairs, each instance in OpenDiscovery is a self-contained, Dockerized scientific challenge. The AI's task is not merely to predict, but to act—to autonomously formulate a hypothesis, conduct experiments and analysis to arrive at new scientific discoveries and their explanations. The immediate, verifiable feedback from this environment provides an ideal training ground for Reinforcement Learning, aiming to elevate AI from knowledgeable assistants to genuine creative partners in science.

Submission Number: 249