Pre-training Pure GNNs as Graph Learners

ICLR 2026 Conference Submission787 Authors

Keywords: graph neural network, pre-training

Abstract: Graphs from different datasets exhibit diverse numbers of features and labels, where each feature or label is associated with different semantic meanings. Such diversity poses challenges in adapting pre-trained graph neural networks (GNNs) to different datasets with a single set of input and output (I/O) module parameters. This raises a fascinating question: Can pure GNNs be pre-trained on diverse datasets, adapting to various datasets effectively without additional effort? To explore this, we propose unified I/O modules that enable pre-training with pure GNNs. Unlike traditional methods that tightly couple parameters to specific datasets, our approach decouples parameters through a shared relation function for the input and uniformly sampled points for the output. These designs effectively resolve the challenges in quantity inconsistency and semantic discrepancies of dataset features and labels. By integrating our I/O modules with various GNN architectures, we demonstrate that pure GNNs can be effective graph learners for direct adaptation to downstream tasks. Pre-training experiments under different setups show that increasing hidden dimensions and the average number of nodes per training dataset enhances model performance. Moreover, fine-tuning the I/O modules with frozen pre-trained graph operators significantly simplifies the model hyperparameter tuning process, achieving superior or comparable performance to supervised models on downstream datasets.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 787