CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks

Seoyeon Choi, Kanghyun Ryu, Jonghoon Ock, Negar Mehr

Keywords: Multi-Agent Reinforcement Learning, Curriculum Learning, Large Language Models

TL;DR: We propose CRAFT, a framework that enables guided exploration in long-horizon multi-robot coordination, where LLMs and VLMs act as coaches to structure curricula and rewards, turning exploration from unguided trial-and-error into structured learning.

Abstract: We propose CRAFT: Coaching Reinforcement learning Autonomously using Foundation models for multi-robot coordination Tasks, a framework that leverages the reasoning capabilities of foundation models to act as a ``coach'' for multi-robot coordination. CRAFT automatically decomposes long-horizon coordination tasks into sequences of subtasks using the planning capability of Large Language Models (LLMs). Then, CRAFT sequentially trains each subtask using reward functions generated by LLM, and refines them through a Vision Language Model (VLM)-guided reward-refinement loop. We evaluate CRAFT on multi-quadruped navigation and bimanual manipulation tasks, demonstrating its capability to learn complex coordination behaviors. In addition, we validate the multi-quadruped navigation policy in real hardware experiments. Project website is \href{https://iconlab.negarmehr.com/CRAFT/}{https://iconlab.negarmehr.com/CRAFT/}

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 19