Unlocking Time Series Foundation Models with Channel Descriptions

Utsav Dutta, Henrik Ohlsson, Sina Khoshfetrat Pakazad, Gerardo Pastrana

Keywords: Representation Learning, Time Series, Deep Learning, JEPA

TL;DR: Leveraging channel descriptions for semantic self-supervised time series representation learning.

Abstract: Traditional time series models are often task-specific and rely on extensive feature engineering. While Transformer-based architectures have advanced sequence modeling in other domains, their use for time series representation learning remains limited. We introduce CHARM, a model that improves representation quality for multivariate time series by incorporating channel-level textual descriptions into its architecture. This design enables the model to exploit contextual information associated with individual sensors while remaining invariant to channel order. CHARM is trained using a Joint Embedding Predictive Architecture (JEPA) with a novel loss function that encourages informative and temporally robust embeddings. Through extensive ablations, we show that integrating channel descriptions significantly enhances representation quality. The learned embeddings yield strong performance across diverse downstream tasks, underscoring the value of description-aware time series modeling.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 21378