COTIC: Embracing Non-uniformity in Event Sequence Data via Multilayer Continuous Convolution
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: temporal point process, time series, continuous convolutions, neural networks
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Massive samples of event sequences occur in various domains, including e-commerce, healthcare, and finance. There are two main challenges regarding modeling such data: methodological and computational. The methodological peculiarity for event sequences is their non-uniformity and sparsity. These requirements make time series models unsuitable. The computational challenge arises from a large amount of available data and the significant length of each sequence. Thus, the problem requires complex and efficient models.
Existing solutions include large recurrent and transformer neural network architectures.
On top of existing blocks, their authors introduce specific intensity functions defined at each moment.
However, due to their parametric nature, these continuous-time-aware intensities represent only a limited class of event sequences.
We propose the COTIC method based on an efficient continuous convolution neural network suitable for the non-uniform occurrence of events in time. In COTIC, dilations and multi-layer architecture efficiently handle long-term dependencies between events. Furthermore, the model provides intensity dynamics in continuous time --- including self-excitement encountered in practice. Being the first to introduce multiple continuous convolution layers that can handle arbitrary complex dependencies via MLP-modeled convolutions, we obtain these properties.
When benchmarked against existing models, the COTIC consistently outperforms them, especially in predicting the next event time and type: it has the average rank of 2.125 vs. 3.688 of the primal competitor. Additionally, its ability to produce effective embeddings showcases its potential for a range of downstream tasks, as produced embeddings are sufficient to solve various downstream tasks, e.g., 0.459 vs. 0.452 baseline accuracy on a 4-label age bin prediction for transactions dataset. The code of the proposed method is available at https://anonymous.4open.science/r/COTIC-F47D/README.md
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8440
Loading