Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning

Elan Sopher Markowitz, Keshav Balasubramanian, Mehrnoosh Mirtaheri, Sami Abu-El-Haija, Bryan Perozzi, Greg Ver Steeg, Aram Galstyan

Keywords: Graph, Learning, Algorithm, Scale, Message Passing, Node Embeddings

Abstract:

Graph Representation Learning (GRL) methods have impacted fields from chemistry to social science. However, their algorithmic implementations are specialized to specific use-cases e.g. "message passing" methods are run differently from "node embedding" ones. Despite their apparent differences, all these methods utilize the graph structure, and therefore, their learning can be approximated with stochastic graph traversals. We propose Graph Traversal via Tensor Functionals (GTTF), a unifying meta-algorithm framework for easing the implementation of diverse graph algorithms and enabling transparent and efficient scaling to large graphs. GTTF is founded upon a data structure (stored as a sparse tensor) and a stochastic graph traversal algorithm (described using tensor operations). The algorithm is a functional that accept two functions, and can be specialized to obtain a variety of GRL models and objectives, simply by changing those two functions. We show for a wide class of methods, our algorithm learns in an unbiased fashion and, in expectation, approximates the learning as if the specialized implementations were run directly.
With these capabilities, we scale otherwise non-scalable methods to set state-of-the-art on large graph datasets while being more efficient than existing GRL libraries -- with only a handful of lines of code for each method specialization.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: GTTF is a meta-algorithm, upon which, many algorithms for graph learning can be implemented, automatically giving them efficiency and scale, yet unbiased learning.

Code: [ isi-usc-edu/gttf](https://github.com/isi-usc-edu/gttf)

Data: [PPI](https://paperswithcode.com/dataset/ppi), [Reddit](https://paperswithcode.com/dataset/reddit)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2102.04350/code)