Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees

Zehong Wang, Zheyuan Zhang, Tianyi Ma, Nitesh V Chawla, Chuxu Zhang, Yanfang Ye

Abstract: Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks---such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.

Lay Summary: Graph-structured data is everywhere---from social networks to molecular structures---but building general-purpose models for graphs has been difficult due to the wide variety of graph types and tasks. Inspired by the success of foundation models in text and vision, this work introduces a new approach to generalize across different graph tasks using a concept called “task-trees.” A task-tree is a structure that captures the essential parts of a graph relevant to a specific task (e.g., classifying a node or predicting a link), and unifies different types of graph tasks into a common format. The paper further proposes a model called GIT (Graph Generality Identifier on Task-Trees), which is pretrained on task-trees from diverse graphs. GIT demonstrates strong performance in fine-tuning, few-shot learning, and even zero-shot generalization across 30+ datasets in five domains. Theoretical analysis supports the effectiveness of task-trees for learning transferable patterns. Overall, this work provides a scalable and principled foundation for training general-purpose graph models, advancing the field toward graph foundation models similar to GPTs for text or CLIP for vision.

Link To Code: https://github.com/Zehong-Wang/GIT

Primary Area: Deep Learning->Foundation Models

Keywords: Graph Neural Network, Foundation Model, Transferability, Theoretical Basis, Tree Structure

Submission Number: 7590