FEAT: A general framework for Feature-aware Multivariate Time-series Representation Learning

Subin Kim, Euisuk Chung, Pilsung Kang

Keywords: multivariate time-series, representation learning, self-supervised learning, contrastive learning, gating, reconstruction

TL;DR: A self-supervised framework for learning feature-aware multivariate time-series representation

Abstract: Multivariate time-series is complex and uncertain. The overall temporal patterns change dynamically over time, and each feature is often observed to have a unique pattern. Therefore, it is challenging to model a framework that can flexibly learn dynamically changing temporal patterns as well as feature-specific unique patterns simultaneously. We propose a general framework for FEature-Aware multivariate Time-series representation learning, called FEAT. Unlike previous methods that only focus on training the overall temporal dependencies, we focus on training feature-agnostic as well as feature-specific patterns in a data-driven manner. Specifically, we introduce a feature-wise encoder to explicitly model the feature-specific information and design an element-wise gating layer that learns the influence of feature-specific patterns per dataset in general. FEAT outperforms the benchmark models in average accuracy on 29 UEA multivariate classification datasets and in MSE and MAE on four multivariate forecasting datasets.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning