TrajGPT: Healthcare Time-Series Representation Learning for Trajectory Prediction

Ziyang Song, Qincheng Lu, He Zhu, David L. Buckeridge, Yue Li

Keywords: Time-series representation learning, healthcare applications, Transformer, ODE, SSM

TL;DR: We propose a novel Transformer model for healthcare time-series representation learning, which is able to make prediction of trajectories from sparse and irregular data.

Abstract: In many domains, such as healthcare, time-series data is irregularly sampled with varying intervals between observations. This creates challenges for classical time-series models that require equally spaced data. To address this, we propose a novel time-series Transformer called **Trajectory Generative Pre-trained Transformer (TrajGPT)**. It introduces a data-dependent decay mechanism that adaptively forgets irrelevant information based on clinical context. By interpreting TrajGPT as ordinary differential equations (ODEs), our approach captures continuous dynamics from sparse and irregular time-series data. Experimental results show that TrajGPT, with its time-specific inference approach, accurately predicts trajectories without requiring task-specific fine-tuning.

Submission Number: 57