Implicit Neural Network on Dynamic Graphs

Yongjian Zhong, Hieu Vu, Tianbao Yang, Bijaya Adhikari

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Implicit Models, Dynamic Graphs

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Recent works have demonstrated that graph convolution neural networks fail either to capture long-range dependencies in the network or suffer from over-smoothing issues. Several recent works have proposed implicit graph neural networks to remedy the issues. However, despite these issues being magnified in dynamic graphs, where the feature aggregation occurs through both the graph neighborhood and across time stamps, no prior work has developed implicit models to overcome these issues. Here we present IDGNN, a novel implicit neural network for dynamic graphs. We demonstrate that IDGNN is well-posed, i.e., it has a unique fixed-point solution. However, the standard iterative algorithm often used to train implicit models is computationally expensive in our setting and cannot be used to train IDGNN efficiently. To overcome this, we pose an equivalent bi-level optimization problem and propose a single-loop training algorithm. We conduct extensive experiments on real-world datasets on both classification and regression tasks to demonstrate the superiority of our approach over the state-of-the-art baseline approaches. We also demonstrate that our bi-level optimization framework maintains the performance of the standard iterative algorithm while obtaining up to 1600x speed-up.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6680