Learning Time-Series Representations by Hierarchical Uniformity-Tolerance Latent Balancing

TMLR Paper5401 Authors

Abstract: We propose TimeHUT, a novel method for learning time-series representations by hierarchical uniformity-tolerance balancing of contrastive representations. Our method uses two distinct losses to learn strong representations with the aim of striking an effective balance between uniformity and tolerance in the embedding space. First, TimeHUT uses a hierarchical setup to learn both instance-wise and temporal information from input time-series. Next, we integrate a temperature scheduler within the vanilla contrastive loss to balance the uniformity and tolerance characteristics of the embeddings. Additionally, a hierarchical angular margin loss enforces instance-wise and temporal contrasts, creating geometric margins between positive and negative pairs of temporal sequences. This approach improves the coherence of positive pairs and their separation from the negatives, enhancing the capture of temporal dependencies within a time-series sample. We evaluate our approach on a wide range of tasks, namely 128 UCR and 30 UAE datasets for univariate and multivariate classification, as well as Yahoo and KPI datasets for anomaly detection. The results demonstrate that TimeHUT outperforms prior methods by considerable margins on classification, while obtaining competitive results for anomaly detection. Finally, detailed sensitivity and ablation studies are performed to evaluate different components and hyperparameters of our method.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=EQPceElWU7&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)

Changes Since Last Submission:

The manuscript desk rejected due to incorrect format, e.g., upper margin. We used the exact TMLR template, but there might have been conflicts among the packages we included with the TMLR style file. Therefore, we commented out the following packages to solve this issue:

\usepackage{wrapfig},
\usepackage{float},
\usepackage{tikz},
\usetikzlibrary{3d},
\usepackage[margin=1in]{geometry},
\usepackage[table]{xcolor}.

Assigned Action Editor: ~Francesco_Locatello1

Submission Number: 5401