Go to NeurIPS 2025 Position Paper Track homepageFalsify, Don’t Just Discover - AI Generated Discoveries are NOT Born Scientific
Keywords: AI-Empowered Falsification, Automated Research, AI Researcher, AI Generated Scientific Discovery, Automated Machine Learning
TL;DR: AI research systems need to explicitly incorporate, design, and optimize automated falsification process to generate scientific discoveries with effectiveness, responsibility, and ethics.
Abstract: Rapid development of artificial intelligence has drastically accelerated the development of scientific discovery. Recently, the rise of Large Language Models (LLMs) has led to the prosperity of autonomous agents, which enable scientists to seek references at different stages of their research. The demonstrated autonomy of these agents has led to designations such as "AI Scientist". However, it remains an open question whether we have truly reached the stage where scientific discovery can be fully automated. In this paper, $\textit{we posit that automated scientific discovery needs \textbf{automated falsification}}$, which has not received sufficient attention in current research favors. As stated in Popper (1935), the central component of scientific research is falsification, where experiments are designed or theories are deduced to validate or refute hypotheses. To automate scientific discovery, the falsification process should also be studied towards full automation. We review the substance of falsification in each stage along the development of AI-accelerated scientific discovery, and analyze the subject, the object, and the degree of automation of the falsification process. Following this, we initiate $\textbf{Baby-AIGS}$, a proof-of-concept AI-generated discovery system enabled by automated falsification. Through qualitative and quantitative studies, we reveal the feasibility of automated falsification, and advocate for responsible and ethical development of such systems for research automation.
Submission Number: 134
Loading