Graph-PDE: Coupled ODE Structure for Graph Neural Networks
Keywords: Graph PDE, Graph ODE, SNNs, GNNs
Abstract: Spike Neural Networks (SNNs), a type of ordinary differential equation (ODE), have evolved as a sophisticated approach for addressing challenges in dynamic graph neural networks. They typically sample binary features at each time step and propagate them using SNNs to achieve spike representation. However, spike ODE remain discrete solutions, limiting their ability to capture continuous changes and subtle dynamics in time series. An effective solution is to incorporate continuous graph ODE into spike ODE to model dynamic graphs from two distinct dimensions, i.e., the time dimension of graph ODE and the latency dimension of spike ODE. The key challenge is to design a structure that seamlessly integrates spike ODE and graph ODE while ensuring the stability of the model. In this paper, we propose Graph-PDE (G-PDE), which combines spike and graph ODE in a unified graph partial differential equation (PDE). Considering the incorporation of high-order structure would preserve more information, alleviating the issue of information loss in first-order ODE. Therefore, we derive the high-order spike representation and propose the second-order G-PDE. Additionally, we prove that G-PDE addresses the issues of exploding and vanishing gradients, making it easier to train deep multi-layer graph neural networks. Finally, we demonstrate the competitive performance of G-PDE compared to state-of-the-art methods across various graph-based learning tasks.
Primary Area: general machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5945
Loading