Causal Reinforcement Learning for Spatio-Temporal Point Processes
Keywords: Spatio-Temporal Point Processes, Reinforcement Learning, Causal Inference
Abstract: Spatio-temporal event sequences are increasingly accessible in various domains such as earthquake forecasting, crime prediction, and healthcare management. These data sources present unique challenges, as they involve both spatial and temporal dimensions, with event sequences exhibiting intricate dependencies over time and space. Neural network-based spatio-temporal point processes offer a sophisticated framework for modeling such event data. Conventional maximum likelihood estimation (MLE) of such data may lead to inaccurate predictions due to model-misspecification and compounding prediction errors. On the other hand, reinforcement learning frameworks, which treat event generation as actions and learn a policy to mimic event generation may alleviate the training/test discrepancy issue. Current reinforcement learning of point processes may have prohibitively poor exploration efficiency. In this paper, we propose the Causal learning improved Reinforcement Learning Spatio-Temporal Point Process (CRLSTPP) framework, which can mitigate the issue of compounding prediction errors and improve exploration efficiency at the same time. Experiments on both synthetic data and real-world data validate the superiority of the proposed model.
Primary Area: learning on time series and dynamical systems
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8086
Loading