(LA)YER-NEIGH(BOR) SAMPLING: DEFUSING NEIGHBORHOOD EXPLOSION

Muhammed Fatih Balin, Umit Catalyurek

Keywords: Graph Neural Networks. Sampling

TL;DR: Paper presents a new sampling algorithm combining layer and neighbor sampling methods

Abstract:

Graph Neural Networks have recently received a significant attention, however, training them at a large scale still remains as a challenge.
Minibatch training coupled with sampling is used to alleviate this challenge.
However existing approaches either suffer from the neighborhood explosion phenomenon or does not have good performance.
To deal with these issues, we propose a new sampling algorithm called LAyer-neighBOR sampling (LABOR).
It is designed to be a direct replacement for Neighborhood Sampling with the same fanout hyperparameter while sampling much fewer vertices, without sacrificing quality.
By design, the variance of the estimator of each vertex matches Neighbor Sampling from the point of view from a single vertex.
In our experiments, we demonstrate the superiority of our approach when it comes to model convergence behaviour against Neighbor Sampling and also the other Layer Sampling approaches under the same limited vertex sampling budget constraints.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/yer-neigh-sampling-defusing-neighborhood/code)