Balancing MSE against Abrupt Changes for Time-Series Forecasting

Junwoo Park, Jungsoo Lee, Youngin Cho, Woncheol Shin, Dongmin Kim, Jaegul Choo, Edward Choi

Keywords: time-series forecasting, data imbalance, loss imbalance, noisy samples

Abstract: Time-series forecasting models often encounter drastic changes in a given period of time (i.e., abrupt changes) which generally occur due to unexpected or unknown events. Despite their scarce occurrences in the training set, abrupt changes incur loss (i.e., MSE) that significantly contributes to the total loss. Therefore, they act as noisy training samples and prevent the model from learning generalizable sequence patterns, namely the normal states. Based on such an intuition, we propose a reweighting framework that down-weights the loss incurred by abrupt changes and up-weights those by normal states. For the reweighting framework, we first define a measurement termed Local Discrepancy (LD) which measures the degree of abruptness of a change in a given period of time. Then, we calculate how frequently the temporal changes appear in the training set based on LD (i.e., estimated LD density). Since normal states generally appear frequently compared to abrupt changes, they achieve higher LD density. Using such a property, we reweight the losses proportionally to the estimated LD density. Our reweighting framework is applicable to existing time-series forecasting models regardless of the architectures. Through extensive experiments on 12 time-series forecasting models over eight datasets with various in-output sequence lengths, we demonstrate that applying our reweighting framework reduces MSE by 10.1% on average and by up to 18.6% in the state-of-the-art model.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning