EveryQuery: Zero-Shot Clinical Prediction via Task-Conditioned Pretraining

Payal Chandak, Gregory Kondas, Isaac S. Kohane, Matthew B.A. McDermott

Keywords: electronic health records, foundation models, zero-shot prediction, task-conditioned pretraining, clinical machine learning, structured data, MIMIC-IV

TL;DR: EveryQuery is an EHR foundation model that takes a (code, duration) query and predicts the outcome in a single forward pass, beating autoregressive baselines on 85% of 200 MIMIC-IV tasks with no degradation on rare events.

Abstract: Foundation models pretrained on electronic health records (EHR) have demonstrated zero-shot clinical prediction. They use an autoregressive procedure to generate synthetic patient futures and aggregate statistics over sampled trajectories, which is computationally expensive, statistically noisy for rare events, and not natively promptable. We introduce EveryQuery, an EHR foundation model that achieves zero-shot inference through task-conditioned pretraining over a structured query language. Rather than generating future trajectories, EveryQuery takes a patient's history and a query $q = (c, \Delta t)$ specifying a target code and a prediction horizon, and directly estimates the outcome probability via a single forward pass. On MIMIC-IV, EveryQuery outperforms an autoregressive baseline on 85% of 200 (code, duration) prediction tasks (mean $\Delta$AUC $+0.17$, 95\% CI $[0.14, 0.19]$). The advantage is most pronounced for rare events as autoregressive performance is strongly coupled to event prevalence, while EveryQuery's is prevalence-invariant. EveryQuery also generalizes to held-out codes and to held-out durations, including extrapolation beyond the longest training horizon. Together, these results suggest that task-conditioned pretraining is a viable paradigm for EHR foundation models.

Submission Number: 113