Improving Accuracy and Generalizability via Multi-Modal Large Language Models Collaboration

Shuili Zhang, Hongzhang Mu, Tingwen Liu

Abstract: With the growing interest in Large Language Models (LLMs), integrating visual tasks has led to the development of Multi-Layer Language Models (MLLMs). Despite their advancements, MLLMs face challenges in accuracy and generalization, often due to resource and time constraints. Addressing these issues, our paper introduces a novel Multi-Agent Collaborative Network for MLLMs (MLLM network). This framework harnesses collective intelligence and cooperation among multiple agents to enhance the accuracy and generalizability of MLLMs. The collaborative nature of our MLLMs—featuring inter-layer neuron interaction and information exchange—facilitates superior processing and integration of multi-modal data. This leads to marked improvements in performance. The findings underscore the efficacy and potential of our proposed framework, presenting a robust solution for complex multi-modal challenges in machine learning and artificial intelligence. Our experimental evaluations demonstrate that this approach significantly surpasses traditional single MLLM architectures in task accuracy and generalization.