SEESAW: Do Graph Neural Networks Improve Node Representation Learning for All?
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Graph Neural Networks, graph machine learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: This paper compared the popular Graph Neural Networks and traditional shallow embedding methods in a unified view, characterized the key drawbacks of GNNs, and proposed a guide for practitioners to improve embedding learning in practice.
Abstract: Graph Neural Networks (GNNs) have manifested significant proficiency in various graph learning tasks over recent years. Owing to their exemplary performance, GNNs have garnered increasing attention from both the research community and industrial practitioners. Consequently, there has been a notable transition away from the conventional and prevalent shallow graph embedding methods. However, in tandem with this transition, an imperative question arises: do GNNs always outperform shallow embedding methods in node representation learning? Despite the doubts cast by multiple recent studies, the field of graph machine learning still lacks a systematic understanding, which is essential for meticulously paving its advancement. To properly answer this question, in this work, we propose a principled framework that unifies the pipelines of representative shallow graph embedding methods and GNNs. With rigorous comparative analysis, we first characterize the primary differences in their design from two different perspectives: the prior of node representation learning, and the neighborhood aggregation mechanism. We then analyze the benefits and drawbacks of using GNNs instead of shallow embedding methods through comprehensive experiments on ten real-world graph datasets. Furthermore, we also empirically validate that our analysis can be generalized to GNNs under various learning paradigms. Armed with these insights, we propose a guide for practitioners in choosing appropriate graph representation learning models under different scenarios.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2077
Loading