Asynchrony Invariance Loss Functions for Graph Neural Networks

Pablo Monteagudo-Lago, Arielle Rosinski, Andrew Joseph Dudzik, Petar Veličković

Track: Proceedings

Keywords: algorithmic reasoning, graph neural networks, asynchrony invariance, asynchronous neural networks

TL;DR: Synchronous GNN computations can result in irrelevant information sent across the graph, potentially hindering algorithmic alignment. The present work introduces self-supervised loss functions to promote invariance to asynchronous execution.

Abstract: A ubiquitous class of graph neural networks (GNNs) operates according to the message-passing paradigm, such that nodes systematically broadcast and listen to their neighbourhood. Yet, these synchronous computations have been deemed potentially sub-optimal as they could result in irrelevant information sent across the graph, thus interfering with efficient representation learning. In this work, we devise self-supervised loss functions biasing learning of synchronous GNN-based neural algorithmic reasoners towards representations that are invariant to asynchronous execution. Asynchrony invariance could successfully be learned, as revealed by analyses exploring the evolution of the self-supervised losses as well as their effect on the learned latent embeddings. Our approach to enforce asynchrony invariance constitutes a novel, potentially valuable tool for graph representation learning, which is increasingly prevalent in multiple real-world contexts.

Submission Number: 10