Fine-Tuning a Time Series Foundation Model with Wasserstein Loss
Keywords: Time Series Foundation Models, Fine-Tuning, Wasserstein Loss, Time Series Evaluation
TL;DR: Applying LLM architectures to the time series forecasting domain by replacing cross-entropy loss with Wasserstein loss
Abstract: Inspired by recent advancements in large language models (LLMs) for Natural Language Processing (NLP), there has been a surge in research focused on developing foundational models for time series forecasting. One approach involves training LLM architectures on tokenized time series data using cross-entropy loss. Although this method has demonstrated promising results, cross-entropy loss is primarily designed for classification tasks and does not account for the distance between classes. To address this limitation, we propose using the Wasserstein loss for such architectures. To validate our approach, we fine-tuned a foundational time series model on $22$ zero-shot datasets, comparing the performance of cross-entropy loss with that of Wasserstein loss. Our results demonstrate that replacing cross-entropy loss with Wasserstein loss significantly improves point estimation.
Submission Number: 60
Loading