Curriculum Dynamic Graph Invariant Learning under Distribution Shift
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Graph Representation Learning, Out Of Distribution Generalization, Curriculum Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Dynamic graph neural networks have attracted intensive research interests recently but generally suffer from handling distribution shifts that widely exist in dynamic graphs. Although the existing works attempt to disentangle the invariant and variant patterns, they ignore the training status of the graph neural network and the importance of training samples at different times, which are critical to model invariant patterns accurately in dynamic graphs under distribution shifts. In this paper, we study distribution shifts in dynamic graphs with curriculum learning for the first time, which remains unexplored and faces the following challenges: (i) how to design a tailored training status evaluation strategy; and (ii) how to design a tailored sample importance reweighting strategy, so as to handle distribution shifts in dynamic graphs. To address these challenges, we propose a Curriculum Dynamic Graph Invariant Learning (CDGIL) model, which can handle distribution shifts in dynamic graphs by capturing and utilizing invariant and variant patterns guided by the proposed curriculum learning strategy. Specifically, we first propose a dual disentangled dynamic attention network to capture the invariant and variant patterns, respectively. Next, we propose a self-paced intervention mechanism based on training status to create adversarial samples by reassembling variant patterns across neighborhoods and time stamps to remove the spurious impacts of variant patterns. Finally, we propose a sample importance reweighting strategy to distinguish invariant and variant patterns better via focusing on the key training samples. Extensive experiments on both synthetic and real-world dynamic graph datasets demonstrate the superiority of our proposed method over state-of-the-art baselines under distribution shifts.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3841
Loading