Reversible and irreversible bracket-based dynamics for deep graph neural networks

Published: 21 Sept 2023, Last Modified: 20 Dec 2023NeurIPS 2023 posterEveryoneRevisionsBibTeX
Keywords: graph neural networks, structure preserving machine learning, neural ordinary differential equations, hamiltonian dynamics, metriplectic dynamics
TL;DR: Novel bracket-inspired GNN architectures are versatile and establish role of reversibility/irreversibility in stability of deep GNNs.
Abstract: Recent works have shown that physics-inspired architectures allow the training of deep graph neural networks (GNNs) without oversmoothing. The role of these physics is unclear, however, with successful examples of both reversible (e.g., Hamiltonian) and irreversible (e.g., diffusion) phenomena producing comparable results despite diametrically opposed mechanisms, and further complications arising due to empirical departures from mathematical theory. This work presents a series of novel GNN architectures based upon structure-preserving bracket-based dynamical systems, which are provably guaranteed to either conserve energy or generate positive dissipation with increasing depth. It is shown that the theoretically principled framework employed here allows for inherently explainable constructions, which contextualize departures from theory in current architectures and better elucidate the roles of reversibility and irreversibility in network performance. Code is available at the Github repository \url{}.
Supplementary Material: pdf
Submission Number: 13826