R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science

ICLR 2026 Conference Submission19663 Authors

Keywords: Data-Driven AI, Multi-Agent System, Large Language Models, Auto Research, LLM, LLM-Agent

TL;DR: We propose R&D-Agent, a flexible framework for exploring LLM-based agent designs, through which we discover an agent system that achieves state-of-the-art performance on MLE-Bench.

Abstract:

Recent advances in AI and ML have transformed data science, yet increasing complexity and expertise requirements continue to hinder progress. Although crowd-sourcing platforms alleviate some challenges, high-level machine learning engineering (MLE) tasks remain labor-intensive and iterative.
We introduce R\&D-Agent, a comprehensive, decoupled, and extensible framework that formalizes the MLE process. R\&D-Agent defines the MLE workflow into two phases and six components, turning agent design for MLE from ad-hoc craftsmanship into a principled, testable process.
Although several existing agents report promising gains on their chosen components, they can mostly be summarized as a partial optimization from our framework's simple baseline.
Inspired by human experts, we designed efficient and effective agents within this framework that achieve state-of-the-art performance.
Evaluated on MLE-Bench, the agent built on R\&D-Agent\ ranks as the top-performing machine learning engineering agent, achieving 35.1\% any medal rate, demonstrating the ability of the framework to speed up innovation and improve accuracy across a wide range of data science applications.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 19663