Reversible and irreversible bracket-based dynamics for deep graph neural networks

Anthony Gruber, Kookjin Lee, Nathaniel Trask

Keywords: graph neural networks, structure preserving machine learning, neural ordinary differential equations, hamiltonian dynamics, metriplectic dynamics

TL;DR: Novel bracket-inspired GNN architectures are versatile and establish role of reversibility/irreversibility in stability of deep GNNs.

Abstract: Recent works have shown that physics-inspired architectures allow the training of deep graph neural networks (GNNs) without oversmoothing. The role of these physics is unclear, however, with successful examples of both reversible (e.g., Hamiltonian) and irreversible (e.g., diffusion) phenomena producing comparable results despite diametrically opposed mechanisms, and further complications arising due to empirical departures from mathematical theory. This work presents a series of novel GNN architectures based upon structure-preserving bracket-based dynamical systems, which are provably guaranteed to either conserve energy or generate positive dissipation with increasing depth. It is shown that the theoretically principled framework employed here allows for inherently explainable constructions, which contextualize departures from theory in current architectures and better elucidate the roles of reversibility and irreversibility in network performance. Code is available at the Github repository \url{https://github.com/natrask/BracketGraphs}.

Submission Number: 13826