Go to ICLR 2026 Conference homepageOn The Expressive Power of GNN Derivatives
Keywords: Graph Neural Networks, GNNs, Expressivity, Message Passing, Geometric deep learning, Differential geometry, Symmetry
TL;DR: We present High Order Derivative-GNN (HOD-GNN), a method that leverages the input derivatives of a base MPNN to enhance the expressive power of a downstream GNN by injecting derivative-informed node features.
Abstract: Despite significant advances in Graph Neural Networks (GNNs), their limited expressivity remains a fundamental challenge. Research on GNN expressivity has produced many expressive architectures, leading to architecture hierarchies with models of increasing expressive power. Separately, derivatives of GNNs with respect to node features have been widely studied in the context of the oversquashing and over-smoothing phenomena, GNN explainability, and more. To date, these derivatives remain unexplored as a means to enhance GNN expressivity. In this paper, we show that these derivatives provide a natural way to enhance the expressivity of GNNs. We introduce High-Order Derivative GNN (HOD-GNN), a novel method that enhances the expressivity of Message Passing Neural Networks (MPNNs) by leveraging high-order node derivatives of the base model. These derivatives generate expressive structure-aware node embeddings processed by a second GNN in an end-to-end trainable architecture. Theoretically, we show that the resulting architecture family's expressive power aligns with the WL hierarchy. We also draw deep connections between HOD-GNN, Subgraph GNNs, and popular structural encoding schemes. For computational efficiency, we develop a message-passing algorithm for computing high-order derivatives of MPNNs that exploits graph sparsity and parallelism. Evaluations on popular graph learning benchmarks demonstrate HOD-GNN’s strong performance on popular graph learning tasks.
Supplementary Material: zip
Primary Area: learning on graphs and other geometries & topologies
Submission Number: 9476
Loading