Time Series as Images: Vision Transformer for Irregularly Sampled Time Series

Zekun Li, Shiyang Li, Xifeng Yan

Keywords: irregularly sampled time series, vision transformer, healthcare, time series classification, multivariate time series

TL;DR: We introduce a new perspective for irregularly sampled time series modeling, i.e., transforming time series data into line graph images and utilizing vision transformers to perform downstream tasks, which achieves SOTA results.

Abstract: Irregularly sampled time series are becoming increasingly prevalent in various domains, especially medical applications. Although different highly-customized methods have been proposed to tackle irregularity, how to effectively model their complicated dynamics and high sparsity is still an open problem. This paper studies the problem from a whole new perspective: transforming irregularly sampled time series into line graph images and adapting powerful vision transformers to perform time series classification in the same way as image classification. Our approach largely simplifies algorithm designs without assuming prior knowledge and can be potentially extended as a general-purpose framework. Despite its simplicity, we show that it substantially outperforms state-of-the-art specialized algorithms on several popular healthcare and human activity datasets. Our code and data are available at \url{https://github.com/Leezekun/ViTST}.

Anonymity: We agree to keep the submission (including any supplements and/or code) anonymous.

Formatting: We confirm that we read and complied with the author's instructions.