TOTEM: Tokenized Time Series Embeddings for General Time Series Analysis
Keywords: representation learning, tokenization, time series
TL;DR: We present TOTEM: a simple, performant, time series tokenizer that works across domains and tasks thereby enabling generalist modeling with strong in-domain and zero-shot performance.
Abstract: The field of general time series analysis has recently begun to explore unified modeling, where a common architectural backbone can be retrained on a specific task for a specific dataset. In this work, we approach unification from a complementary vantage point: unification of time series data representations across domains in many tasks. To this end, we explore the impact of discrete, learnt, time series data representations that enable generalist, cross-domain training. Our method, TOTEM, or Tokenized Time Series Embeddings, proposes a simple tokenizer architecture that embeds time series data from varying domains using a discrete vectorized representation learned in a self-supervised manner. TOTEM works across multiple tasks and domains with minimal to no tuning. We study the efficacy of TOTEM with an extensive evaluation on 17 real world time series datasets across 3 tasks. We evaluate both the specialist (i.e., training a model on each domain) and generalist (i.e., training a single model on many domains) settings, and show that TOTEM matches or outperforms previous best methods on several popular benchmarks. Please find the full paper here: https://arxiv.org/pdf/2402.16412.pdf, and the code here: https://github.com/SaberaTalukder/TOTEM.
Primary Subject Area: Domain specific data issues
Paper Type: Research paper: up to 8 pages
DMLR For Good Track: Participate in DMLR for Good Track
Participation Mode: In-person
Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.
Submission Number: 80
Loading