Partial Channel Dependence with Channel Masks for Time Series Foundation Model

Seunghan Lee, Taeyoung Park, Kibok Lee

Keywords: Time Series, Foundation Model

TL;DR: We introduce the concept of partial channel dependence (PCD) to partially adjust channel dependence (CD) in time series using a proposed channel mask constructed from dataset-specific information.

Abstract: While advances in foundation models have extended to the time series domain, they have primarily focused on designing model architectures to address external heterogeneity between datasets, e.g., varying numbers of channels, often overlooking internal heterogeneity, e.g., varying channel dependencies. In this work, we introduce the concept of partial channel dependence (PCD), which enables a more sophisticated adjustment of channel dependencies based on dataset-specific information. To achieve PCD, we propose a channel mask that captures the relationships between channels within a dataset using two key components: 1) a correlation matrix that encodes relative dependencies between channels, and 2) domain parameters that learn the absolute dependencies specific to each dataset, refining the correlation matrix. We validate the effectiveness of our method across various tasks, including forecasting, classification, imputation, and anomaly detection.

Submission Number: 8