CoLES: Contrastive learning for event sequences with self-supervision

Dmitrii Babaev, Nikita Ovsov, Ivan A Kireev, Gleb Gusev, Maria Ivanova, Alexander Tuzhilin

Keywords: representation learning, contrastive learning, neural networks, event sequiences

Abstract: We address the problem of self-supervised learning on discrete event sequences generated by real-world users. Self-supervised learning incorporates complex information from the raw data in low-dimensional fixed-length vector representations that could be easily applied in various downstream machine learning tasks. In this paper, we propose a new method CoLES, which adopts contrastive learning, previously used for audio and computer vision domains, to the discrete event sequences domain in a self-supervised setting. Unlike most previous studies, we theoretically justify under mild conditions that the augmentation method underlying CoLES provides representative samples of discrete event sequences. We evaluated CoLES on several public datasets and showed that CoLES representations consistently outperform other methods on different downstream tasks.

One-sentence Summary: We propose a new method CoLES, which adapts self-supervised contrastive learning, to the discrete event sequence domain

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=R-pNI8vreR