E-ProTran: Efficient Probabilistic Transformers for Forecasting
Keywords: Time Series Forecasting, Transformer, Latent Variable Model
TL;DR: This paper introduces E-ProTran, a redesigned transformer model for probabilistic time series forecasting. E-ProTran maintains high performance while improving efficiency with simplified attention layers.
Abstract: Time series forecasting involves predicting future data points based on historical patterns and is critical for applications in fields such as healthcare, financial markets, and weather forecasting, where scalability and efficiency, particularly in training and inference times, are paramount. Transformers, known for their ability to handle long-range dependencies in sequential data, have shown promise in time series analysis. However, the complexity of transformer models can lead to overparameterization, extended training times, and scalability challenges, which can become even more problematic if the assumptions of the underlying generative model are overly complicated. In this paper, we introduce E-ProTran by re-designing a state-of-the-art transformer for probabilistic time series forecasting. We empirically demonstrate that E-ProTran maintains high performance while significantly enhancing efficiency without necessarily reconstructing the conditioned history. Our model incorporates simplified attention layers and design adjustments that reduce computational overhead without compromising accuracy, offering a more efficient and scalable solution for time series forecasting.
Submission Number: 84
Loading