How to Find Your Friendly Neighborhood: Graph Attention Design with Self-SupervisionDownload PDF

Published: 12 Jan 2021, Last Modified: 05 May 2023ICLR 2021 PosterReaders: Everyone
Keywords: Graph Neural Network, Attention Mechanism, Self-supervised Learning
Abstract: Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when graphs are noisy. In this paper, we propose a self-supervised graph attention network (SuperGAT), an improved graph attention model for noisy graphs. Specifically, we exploit two attention forms compatible with a self-supervised task to predict edges, whose presence and absence contain the inherent information about the importance of the relationships between nodes. By encoding edges, SuperGAT learns more expressive attention in distinguishing mislinked neighbors. We find two graph characteristics influence the effectiveness of attention forms and self-supervision: homophily and average degree. Thus, our recipe provides guidance on which attention design to use when those two graph characteristics are known. Our experiment on 17 real-world datasets demonstrates that our recipe generalizes across 15 datasets of them, and our models designed by recipe show improved performance over baselines.
One-sentence Summary: We propose a method that self-supervise graph attention through edges and it should be designed according to the average degree and homophily of graphs.
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
Code: [![github](/images/github_icon.svg) dongkwan-kim/SuperGAT](https://github.com/dongkwan-kim/SuperGAT) + [![Papers with Code](/images/pwc_icon.svg) 1 community implementation](https://paperswithcode.com/paper/?openreview=Wi5KUNlqWty)
Data: [Citeseer](https://paperswithcode.com/dataset/citeseer), [Cora](https://paperswithcode.com/dataset/cora), [OGB](https://paperswithcode.com/dataset/ogb), [PPI](https://paperswithcode.com/dataset/ppi), [Pubmed](https://paperswithcode.com/dataset/pubmed), [Wiki-CS](https://paperswithcode.com/dataset/wiki-cs)
11 Replies

Loading