M3Builder: A Multi-Agent System for Automated Machine Learning in Medical Imaging

ACL ARR 2025 May Submission3081 Authors

Abstract:

In this paper, we aim to automate the labor-intensive process of developing machine learning~(ML)-based tools for medical imaging, paving the way for the self-evolution of medical agentic systems.
(i) We present M^3Builder, a multi-agent collaboration architecture designed to automate ML in medical imaging, that divide-and-conquers complex medical ML with four specialized agents.
(ii) To better fit in the professional medical imaging domain, we build up a medical imaging specialized ML context protocol, a structured environment designed to provide agents with comprehensive free-text descriptions of medical datasets, training code templates, and interaction tools.
(iii) To monitor the progress, we propose M^3Bench, spanning four medical imaging ML tasks across 14 datasets, covering both 2D and 3D data. In experiments, we demonstrate that, using the same agent core, M^3Builder outperforms existing automated ML agentic architectures by achieving a higher ML task completion rate, achieving a 94.29% success rate with acceptable model performance. This highlights the promising potential for fully automated ML-based tool development in medical imaging. Code will be available on Github upon publication.

Paper Type: Long

Research Area: Language Modeling

Research Area Keywords: LLM/AI agents,applications,prompting

Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models

Languages Studied: English

Submission Number: 3081