ROAM: A Relation-aware Optimal Transport-based Adaptive Mixture-of-Expert-Group Framework for Multimodal Knowledge Graph Completion

Xinrong Hu, Bin Li, Wanqing Li, Jie Yang, Wenbin Zhang, Yi Guo

Keywords: Multimodal Knowledge Graph, Mixture-of-Expert-Groups, Optimal Transport, Conditional Aware Optimal Transport

Abstract: Multimodal Knowledge Graph Completion (MMKGC) aims to predict missing facts by reasoning over heterogeneous information sources, including structural, textual, and visual modalities. A central challenge in this task lies in effectively integrating modality-specific information while preserving their distinct semantics and mitigating cross-modal interference. Recent efforts have explored employing Mixture of-Experts (MoE) architectures to address this issue. However, many of these approaches rely on predefined expert routing or task-agnostic distance measures, thereby limiting their adaptability and performance. This paper proposes ROAM, a Relation-aware Optimal Transport-based Adaptive Mixture-of-Expert-Group framework, which dynamically routes modality-specific embeddings across expert groups conditioned on relation semantics. Specifically, ROAM first establishes modality-specialized expert groups to disentangle representation learning across modalities. Then, an Optimal Transport-based gating mechanism is introduced with a learnable, relation-conditioned cost function. Expert groups are further represented dynamically via their constituent parameters, enabling context-sensitive routing and capturing relation-aware specialization. Extensive experiments on multiple MMKGC benchmarks demonstrate that ROAM achieves state-of-the-art performance, achieving up to 9.76% relative gains.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 15302