Uncertainty prediction and calibration using multi-expert gating mechanism

Kun Cao, Zongxia Xie

Abstract: Time series prediction is applied in many fields as a fundamental task. Although existing methods have achieved satisfactory accuracy, making their prediction more credible is still a significant challenge. In the existing research, the uncertainty estimate method is relatively mature. However, the uncertainty is generally calibrated with post-processing so that the prediction probability of the model matches the actual likelihood. In this paper, we present a novel approach to predict and calibrate the uncertainty of existing state-of-the-art time series models. We regularize the model through multi-expert gating mechanisms while taking advantage of the differences between experts to build new uncertainty estimation methods and propose the AU-Loss function, which combines accuracy and uncertainty terms. Experiments across different real-world datasets with various state-of-the-art time series models show that our method can effectively improve the calibration effect and credibility for regression problems.