Towards Multimodal Inductive Learning: Adaptively Embedding MMKG via Prototypes

Shundong Yang, Jing Yang, Xiaowen Jiang, Yuan Gao, Laurence T. Yang, Ruikun Luo, Jieming Yang

Abstract: Multimodal Knowledge Graphs (MMKG) models integrate multimodal contexts to improve link prediction performance. All existing MMKG models follow the transductive setting with a fixed predefined set, meaning that all the entities, relations, and multimodal information in the test graph are observed during training. This hinders their generalization to real-world MMKG with unseen entities and relations. Intuitively, a MMKG model trained on DBpedia cannot infer on Freebase. To address above limitations, we make the first attempt towards inductive learning for MMKG and propose a multimodal <u>Ind</u>uctive <u>MMKG</u> model (IndMKG) that is <u>universal</u> and transferable to any MMKG. Distinct from existing transductive methods, our model does not rely on specific trained embeddings; instead, IndMKG generates adaptive embeddings conditioned on any new MMKG via multimodal prototypes. Specifically, we construct class-adaptive prototypes to appropriately characterize the multimodal feature distribution of the given graph and equip IndMKG with robust adaptability to multimodal information across MMKGs. In addition, IndMKG learns non-specific structural embeddings based on meta relations. Such strategies tackle the challenge of notable multimodal feature discrepancies in cross-graph induction and allow the pre-trained IndMKG model to effectively zero-shot generalize to any MMKG. The strong performance in both inductive and transductive settings, across more than 20+ different scenarios, confirms the effectiveness and robustness of IndMKG. Our code is released at https://github.com/MMKGer/IndMKG/.

External IDs:dblp:conf/www/YangYJGYLY25