CAML: Collaborative Auxiliary Modality Learning for Multi-Agent Systems
Keywords: Collaborative Auxiliary Modality Learning, Multi-Agent Collaboration, Cross-Modality Knowledge Distillation
Abstract: Multi-modality learning has become a crucial technique in enhancing the performance of machine learning applications across various domains, including autonomous driving, robotics, and perception systems. Existing frameworks, such as Auxiliary Modality Learning (AML), effectively utilize multiple data sources during training and enable inference with reduced modalities, but they primarily operate in a single-agent context. This limitation is particularly critical in dynamic environments, such as connected autonomous vehicles (CAV), where incomplete data coverage can result in decision-making blind spots. To address these challenges, we introduce Collaborative Auxiliary Modality Learning ($\textbf{CAML}$), a novel extension of the AML framework for multi-agent systems. $\textbf{CAML}$ facilitates collaboration among agents by allowing them to share multimodal data during training. During inference, each agent operates effectively with fewer modalities, ensuring robustness in performance even with missing data. We analyze the effectiveness of $\textbf{CAML}$ from the perspective of uncertainty reduction and data coverage, providing a theoretical support to understand and explain why $\textbf{CAML}$ works better than AML. We then validate $\textbf{CAML}$ through experiments in collaborative decision-making for CAV in accident-prone scenarios. Experimental results show that $\textbf{CAML}$ outperforms AML across all tested scenarios, achieving up to a ${\bf 58.3}$% improvement in accident detection. Additionally, we validate our approach on real-world data from aerial-ground vehicles for collaborative semantic segmentation, achieving up to ${\bf 10.8}$% improvement in mIoU compared to AML.
Primary Area: other topics in machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 12682
Loading