Relation-Aware Graph Foundation Model for Few-shot Learning

ICLR 2026 Conference Submission17099 Authors

Keywords: Graph foundation models, Graph neural networks

Abstract: In recent years, large language models (LLMs) have demonstrated remarkable ability to generalize across diverse natural language processing (NLP) tasks. Inspired by this success, graph foundation models (GFMs) have emerged as a promising direction in graph learning, aiming to achieve cross-dataset generalization through large-scale pre-training. However, unlike language models that rely on explicit token representations, graphs lack a well-defined unit for generalization, making it challenging to design effective pre-training strategies. In this work, we propose REEF, a novel GFM framework that leverages relation tokens as the fundamental units. Analogous to token vocabularies in LLMs, we construct a vocabulary of relation tokens to encode relational information within graphs. To accommodate diverse relations, we introduce two hypernetworks that adaptively generate the parameters of aggregators and classifiers in graph neural networks based on relation tokens. In addition, we design another hypernetwork to construct dataset-specific projectors and incorporate a dataset-level feature bias into the initial node representations, enhancing flexibility across different datasets with the same relation. Furthermore, we adopt graph data augmentation and a mixed-dataset pre-training strategy, allowing REEF to capture relational diversity more effectively. Extensive experiments show that REEF consistently outperforms existing methods on both pre-training and transfer learning tasks, and demonstrates strong generalization in few-shot transfer scenarios, underscoring its potential as a powerful foundation model for graph-based applications.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 17099