Enhancing Multi-Agent Multi-Modal Collaboration with Fine-Grained Reward Modeling

Qian Yang, Weixiang Yan, Aishwarya Agrawal

Keywords: Multi-Agent Collaboration, Decomposition, VQA

TL;DR: We use fine-grained reward modeling to enhance multi-agent collaboration under multi-modal environment.

Abstract:

Multi-Modal Large Language Models (MLLMs) have significantly advanced multi-modal reasoning but still struggle with compositional reasoning tasks. Multi-agent collaboration provides a promising solution by leveraging the distinct capabilities of different agents. Specifically, a decomposer agent to handle task breakdown and an answerer agent to generate responses.
While there have been efforts to adaptively decompose tasks based on the answerer agent's capabilities, such as using in-context learning, these methods often prove insufficient for fully effective decomposition.
We address this issue by enhancing collaboration through fine-grained reward modeling, where each generated sub-question is assigned a specialized reward without requiring extra annotation or tuning of a reward model.
Our proposed method dynamically optimizes the decomposition process, enabling better alignment between agents. Experimental results on four vision-language tasks demonstrate consistent improvements, with a 5.5\% absolute increase in mean performance over traditional approaches. These findings highlight the efficacy of fine-grained reward modeling for enhancing multi-agent, multi-modal collaboration.

Submission Number: 46