Semi-Supervised End-To-End Contrastive Learning For Time Series Classification
Keywords: Semi-supervised, Contrastive learning, Time series, EEG
Abstract: Time series classification is a critical task in various domains, such as finance, healthcare, and sensor data analysis. Unsupervised contrastive learning has garnered significant interest in learning effective representations from time series data with limited labels. The prevalent approach in existing contrastive learning methods consists of two separate stages: pre-training the encoder on unlabeled datasets and fine-tuning the well-trained model on a small-scale labeled dataset. However, such two-stage approaches suffer from several shortcomings, such as the inability of unsupervised pre-training contrastive loss to directly affect downstream fine-tuning classifiers, and the lack of exploiting the classification loss which is guided by valuable ground truth. In this paper, we propose an end-to-end model called SLOTS (Semi-supervised Learning fOr Time clasSification). SLOTS receives semi-labeled datasets, comprising a large number of unlabeled samples and a small proportion of labeled samples, and maps them to an embedding space through an encoder. We calculate not only the unsupervised contrastive loss but also measure the supervised contrastive loss on the samples with ground truth. The learned embeddings are fed into a classifier, and the classification loss is calculated using the available true labels. The unsupervised, supervised contrastive losses, and classification loss are jointly used to optimize the encoder and classifier. We evaluate SLOTS by comparing it with ten state-of-the-art methods across five datasets. On an EEG-based emotion recognition task using the DEAP dataset with only 10% labeled data, SLOTS significantly outperforms two-stage baselines, achieving up to a 16.10% higher F1 score (compared to TS-TCC) and a 38.49% higher absolute accuracy (compared to TS2Vec) when the labeling ratio increases to 100%. SLOTS also attains the best performance on four diverse datasets with an average 3.55% margin in F1. In various evaluation setups, including leave-trials-out and leave-subjects-out, SLOTS consistently achieves top performance. The results demonstrate that SLOTS is a simple yet effective framework. When compared to the two-stage framework, our end-to-end SLOTS utilizes the same input data, consumes a similar computational cost, but delivers significantly improved performance. Crucially, our end-to-end framework is model-agnostic, allowing it to be seamlessly integrated with any existing self-supervised contrastive model in order to enhance its performance. We release code and datasets at https://anonymous.4open.science/r/SLOTS-242E.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3448
Loading