MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs
Keywords: Graph neural network, prompting, pre-training, multi-task, few-shot learning
TL;DR: We propose MultiGPrompt, a multi-task pre-training and prompting framework for few-shot learning on graphs.
Abstract: Graphs can inherently model interconnected objects on the Web, thereby facilitating a series of Web applications, such as web analyzing and content recommendation. Recently, Graph Neural Networks (GNNs) have emerged as a mainstream technique for graph representation learning. However, their efficacy within an end-to-end supervised framework is significantly tied to the availability of task-specific labels. To mitigate labeling costs and enhance robustness in few-shot settings, pre-training on self-supervised tasks has emerged as a promising method, while prompting has been proposed to further narrow the objective gap between pretext and downstream tasks.
Although there has been some initial exploration of prompt-based learning on graphs, they primarily leverage a single pretext task, resulting in a limited subset of general knowledge that could be learned from the pre-training data.
Hence, in this paper, we propose MultiGPrompt, a novel multi-task pre-training and prompting framework to exploit multiple pretext tasks for more comprehensive pre-trained knowledge. First, in pre-training, we design a set of *pretext tokens* to synergize multiple pretext tasks. Second, we propose a dual-prompt mechanism consisting of *composed* and *open* prompts to leverage task-specific and global pre-training knowledge, to guide downstream tasks in few-shot settings.
Finally, we conduct extensive experiments on six public datasets to evaluate and analyze MultiGPrompt.
Track: Graph Algorithms and Learning for the Web
Submission Guidelines Scope: Yes
Submission Guidelines Blind: Yes
Submission Guidelines Format: Yes
Submission Guidelines Limit: Yes
Submission Guidelines Authorship: Yes
Student Author: Yes
Submission Number: 622
Loading