[AML]Systematic Idea Refinement for Machine Learning Research Agents
Keywords: Automatic Machine Learning, Automatic Research, Research Idea Generation, LLM, Multi-Agent System
TL;DR: We propose to automatically generate and refine research idea as guidelines for ML research agents via a plug-and-play multi-agent method.
Abstract: The rapid evolution of large language models (LLMs) has catalyzed new methodologies for automating machine learning (ML) research, particularly in the areas of code generation and model design. While LLM-empowered agents have demonstrated notable success in generating syntactically correct solutions, they often fall short in addressing the complexity of real-world ML tasks, resulting in solutions that may be technically sound but suboptimal in terms of performance and adaptability.
To address this gap, this project introduces a multi-agent framework named $\textbf{Baby-AIGS-MLer}$ designed to enhance ML research agents in a pluggable way under existing Baby-AIGS framework. The proposed system integrates multiple agents, each with a specialized role, that work together to iteratively propose, refine, and evaluate diverse research ideas. Baby-AIGS-MLer leverages multi-level feedback from different stages of the research process, including LLM-based judgments, experimental results, and code generation, to expand the solution space and improve performance, which allows ideas to evolve based on both theoretical insights and empirical evidence. The effectiveness of Baby-AIGS-MLer is evaluated using MLE-Bench, a comprehensive benchmark suite consisting of $\textit{75 Kaggle competitions}$ across various ML tasks. Our preliminary experimental results show that the proposed multi-agent system significantly improves both the performance and efficiency of ML research agents, enabling them to generate and refine more optimal solutions. By incorporating systematic idea exploration and refinement, this approach paves the way for more advanced, self-improving agents that can contribute to the evolving landscape of machine learning research.
Submission Number: 17
Loading