UniGM: Unifying Multiple Pre-trained Graph Models via Adaptive Knowledge Aggregation

Jintao Chen, Fan Wang, Shengye Pang, Siwei Tan, Mingshuai Chen, Tiancheng Zhao, Meng Xi, Jianwei Yin

Abstract: Recent years have witnessed remarkable advances in graph representation learning using Graph Neural Networks (GNNs). To fully exploit the unlabeled graphs, researchers pre-train GNNs on large-scale graph databases and then fine-tune these pre-trained Graph Models (GMs) for better performance in downstream tasks. Because different GMs are developed with diverse pre-training tasks or datasets, they can be complementary to each other for a more complete knowledge base. Naturally, a compelling question is emerging: How can we exploit the diverse knowledge captured by different GMs simultaneously in downstream tasks? In this paper, we make one of the first attempts to exploit multiple GMs to advance the performance in the downstream tasks. More specifically, for homogeneous GMs that share the same model architecture but are obtained with different pre-training tasks or datasets, we align each layer of these GMs and then aggregate them adaptively on a per-sample basis with a tailored Recurrent Aggregation Policy Network (RAPNet). For heterogeneous GMs with different model architectures, we design an alignment module to align the output of diverse GMs and a meta-learner to decide the importance of each GM conditioned on each sample automatically before aggregating the GMs. Extensive experiments on various downstream tasks from 3 domains reveal our dominance over each single GM. Additionally, our methods (UniGM) can achieve better performance with moderate computational overhead compared to alternative approaches including ensemble and model fusion. Also, we verify that our methods are not limited to graph data but could be flexibly applied to image and text data. The codes can be seen in the anonymous link: https://anonymous.4open.science/r/UniGM-DA65.

Relevance To Conference: For multimodal data, there may be correlations and dependencies between different modalities, and information fusion and propagation between modalities can be achieved by graph neural networks to obtain richer and more comprehensive representations. The method proposed in this work can facilitate the fusion of multimodal data and pre-trained models.

Primary Subject Area: [Content] Multimodal Fusion

Submission Number: 2245