TransPL: VQ-Code Transition Matrices for Pseudo-Labeling of Time Series Unsupervised Domain Adaptation

Jaeho Kim, Seulki Lee

TL;DR: Construct VQ code transition matrix for time series unsupervised domain adaptation

Abstract: Unsupervised domain adaptation (UDA) for time series data remains a critical challenge in deep learning, with traditional pseudo-labeling strategies failing to capture temporal patterns and channel-wise shifts between domains, producing sub-optimal pseudo labels. As such, we introduce TransPL, a novel approach that addresses these limitations by modeling the joint distribution $P(X,y)$ of the source domain through code transition matrices, where the codes are derived from vector quantization (VQ) of time series patches. Our method constructs class- and channel-wise code transition matrices from the source domain and employs Bayes' rule for target domain adaptation, generating pseudo-labels based on channel-wise weighted class-conditional likelihoods. TransPL offers three key advantages: explicit modeling of temporal transitions and channel-wise shifts between different domains, versatility towards different UDA scenarios (e.g., weakly-supervised UDA), and explainable pseudo-label generation. We validate TransPL's effectiveness through extensive analysis on four time series UDA benchmarks and confirm that it consistently outperforms state-of-the-art pseudo-labeling methods by a strong margin (6.1\% accuracy improvement, 4.9\% F1 improvement), while providing interpretable insights into the domain adaptation process through its learned code transition matrices.

Lay Summary:

We utilized a method to convert complex time series data into simple discrete tokens (like letters in an alphabet) using a technique called VQVAE. This transformation allows us to study how these tokens transition from one to another over time, creating unique transition patterns for each time series. By aggregating all the token patterns from a user, we can construct a transition matrix to represent a user.

By calculating these transition probabilities between tokens, we created a mathematical "fingerprint" for each user's data. When we compared these fingerprints between different users, we discovered that some properties are shared while others are unique. This insight allowed us to develop a pseudo-labeling approach where we can use one user's fingerprint to classify another user's time series data.

Simply put, when two fingerprints share significant similarities, we can reasonably conclude that the underlying time series belong to the same class. Conversely, when fingerprints differ substantially, we can confidently determine they represent different classes. This approach provides a powerful method for constructing pseudo labels when traditional labeled data is limited or unavailable.

Link To Code: https://github.com/eai-lab/TransPL

Primary Area: Deep Learning->Sequential Models, Time series

Keywords: Time Series, Transition Matrices, VQVAE, Adaptation

Submission Number: 2287