LightCTS*: Lightweight Correlated Time Series Forecasting Enhanced With Model Distillation

Zhichen Lai, Dalin Zhang, Huan Li, Christian S. Jensen, Hua Lu, Yan Zhao

Abstract: Correlated time series (CTS) forecasting is essential in many practical applications, such as traffic management and server load control. Various deep learning based solutions have been proposed to improve forecasting accuracy. However, while models have become increasingly computationally intensive, they struggle to improve accuracy. This study aims instead to enable more lightweight, accurate models suitable for resource-constrained devices. To achieve this goal, we characterize popular CTS forecasting models, yielding two observations for developing lightweight CTS forecasting. On this basis, we propose the LightCTS framework that adopts plain stacking of temporal and spatial operators instead of alternate stacking which is much more computationally expensive. Moreover, LightCTS features light temporal and spatial operators, L-TCN and GL-Former, offering improved computational efficiency without compromising their feature extraction capabilities. LightCTS also encompasses a last-shot compression scheme to reduce redundant temporal features and speed up subsequent computations. Next, we equip LightCTS with two knowledge distillation modules, Tafd and Caad, that result in LightCTS$^\star$ retaining the original benefits of LightCTS, while also being able to adapt to varying levels of ultra-constrained resources. Experimental studies offer detailed insight into these proposals and provide evidence that both LightCTS and LightCTS$^\star$ are capable of nearly state-of-the-art accuracy at substantially reduced computational costs.