Attention-Based Feature Online Conformal Prediction for Time Series

TMLR Paper9587 Authors

Abstract: Online conformal prediction (OCP) wraps around any pre-trained predictor to produce prediction sets with coverage guarantees that hold irrespective of temporal dependencies or distribution shifts. However, standard OCP faces two key limitations: it operates in the output space using simple nonconformity (NC) scores, and it treats all historical observations uniformly when estimating quantiles. This paper introduces attention-based feature OCP (AFOCP), which addresses both limitations through two key innovations. First, AFOCP operates in the feature space of pre-trained neural networks, leveraging learned representations to construct more compact prediction sets by concentrating on task-relevant information while suppressing nuisance variation. Second, AFOCP incorporates a multi-head attention mechanism that adaptively weights historical observations based on their relevance to the current test point, effectively handling non-stationarity and distribution shifts. We provide theoretical guarantees showing that AFOCP maintains long-term coverage while achieving smaller long-term time-averaged prediction sets than standard OCP under mild regularity conditions. Extensive experiments on synthetic and real-world time series datasets demonstrate that AFOCP consistently reduces the prediction interval lengths by as much as $88\%$ relative to OCP and yields shorter intervals than the online counterparts of representative offline CP designs for time series, while maintaining target coverage levels, validating the benefits of both feature-space calibration and attention-based adaptive weighting.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Min_Wu2

Submission Number: 9587