WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales

Drew Prinster, Xing Han, Anqi Liu, Suchi Saria

TL;DR: For monitoring the post-deployment risks of AI/ML models (to adapt to mild shifts and detect severe ones), we propose and evaluate a weighted generalization of conformal test martingales

Abstract: Responsibly deploying artificial intelligence (AI) / machine learning (ML) systems in high-stakes settings arguably requires not only proof of system reliability, but also continual, post-deployment monitoring to quickly detect and address any unsafe behavior. Methods for nonparametric sequential testing---especially conformal test martingales (CTMs) and anytime-valid inference---offer promising tools for this monitoring task. However, existing approaches are restricted to monitoring limited hypothesis classes or ``alarm criteria'' (e.g., detecting data shifts that violate certain exchangeability or IID assumptions), do not allow for online adaptation in response to shifts, and/or cannot diagnose the cause of degradation or alarm. In this paper, we address these limitations by proposing a weighted generalization of conformal test martingales (WCTMs), which lay a theoretical foundation for online monitoring for any unexpected changepoints in the data distribution while controlling false-alarms. For practical applications, we propose specific WCTM algorithms that adapt online to mild covariate shifts (in the marginal input distribution), quickly detect harmful shifts, and diagnose those harmful shifts as concept shifts (in the conditional label distribution) or extreme (out-of-support) covariate shifts that cannot be easily adapted to. On real-world datasets, we demonstrate improved performance relative to state-of-the-art baselines.

Lay Summary:

Responsibly deploying artificial intelligence (AI) systems in high-stakes settings—such as to assist doctors in medical diagnosis or AI chatbot agents used at internet-wide scale—requires continual monitoring of the AI’s performance, to quickly detect and address any unsafe behavior. It turns out, however, that AI monitoring is very difficult: many current approaches are too slow to detect truly harmful changes, or they may raise many false alarms and so risk creating a “boy who cried wolf” effect.

To address these challenges, we introduce a novel framework called “WATCH” (**W**eighted **A**daptive **T**esting for **C**hangepoint **H**ypotheses), which monitors AI deployments with three main goals:

    (1) **Detection:** First, WATCH quickly detects harmful shifts, and it catches them faster than directly tracking standard risk metrics.

    (2) **Adaptation:** Second, WATCH adapts in real-time to mild or benign changes (which are commonplace but not harmful) to reduce unnecessary alarms and improve the utility of the AI’s outputs for end-users.

    (3) **Root-Cause Analysis:** Lastly, WATCH diagnoses the cause of degradation (ie, as a severe change in the data inputs or a fundamental “concept shift” in input-output relationships) to inform appropriate recovery (eg, retraining) of the AI.

WATCH charts a path toward safer and more responsible deployment of AI systems via adaptive monitoring.

Link To Code: https://github.com/aaronhan223/watch

Primary Area: General Machine Learning->Evaluation

Keywords: AI Safety Monitoring, Conformal Martingale, Conformal Prediction, Changepoint Detection, Sequential Testing, Adaptive Testing, Anytime-Valid Inference, E-values, E-processes, Distribution Shift

Submission Number: 9679