Time-LlaMA: Adapting Large Language Models for Time Series Modeling via Dynamic Low-rank Adaptation

Ellen Yi Ge, Juyuan Zhang, Wei Zhu, Hui Yi Leong, Jiechao Gao

Keywords: large languag model, time series modeling, modality alignment, low-rank adaptation

Abstract: Time series modeling holds significant importance in many industrial applications and has been extensively studied. A series of recent studies have demonstrated that large language models (LLMs) possess robust pattern recognition and semantic understanding capabilities over time series data. However, the current literature have yet striked a high-quality balance between (a) effectively aligning the time series and natural language modalities and (b) keeping the inference efficiency for industrial deployment. To address the above issues, we now propose the Time-LlaMA framework. Time-LlaMA first converts the time series input into token embeddings through a linear tokenization mechanism. Second, the time series token embeddings are aligned with the text prompts. Third, to further adapt the large languag model (LLM) backbone for time series modeling, we have developed a dynamic low-rank adaptation technique (DynaLoRA). DynaLoRA dynamically chooses the most suitable LoRA modules at each layer of the Transformer backbone for each time series input, enhancing the model's predictive capabilities. Our experimental results on an extensive collection of challenging open and proprietary time series tasks confirm that our proposed method achieves the state-of-the-art (SOTA) performance and have potentials for wide industrial usages.\footnote{Codes will be made public upon acceptance. }

Archival Status: Archival

Paper Length: Long Paper (up to 8 pages of content)

Submission Number: 330