Amortized In-Context Mixed Effect Transformer Models: A Zero-Shot Approach for Pharmacokinetics

Cesar Ojeda, Ramses J Sanchez, Wilhelm Huisinga, Niklas Hartung

TL;DR: Transformer based amortized inference for mix effect models

Abstract: Accurate dose-response forecasting under sparse sampling is central to precision pharmacotherapy. We present the Amortized In-Context Mixed-Effect Transformer (AICMET) model, a transformer‑based, latent‑variable framework that unifies mechanistic compartmental priors with amortized, in‑context Bayesian inference. AICMET is *pre‑trained* on hundreds of thousands of synthetic pharmacokinetic trajectories with Ornstein-Uhlenbeck priors over the parameters of compartment models, endowing the model with strong inductive biases and enabling *zero‑shot adaptation* to new compounds. At inference time, AICMET is *conditioned on the collective context of previously profiled trial participants*, generating calibrated posterior predictions for newly enrolled patients after a few early drug concentration measurements. This capability collapses traditional model development cycles from weeks to seconds, while preserving some degree of expert modelling. Experiments across public datasets show that AICMET attains state‑of‑the‑art predictive accuracy, and faithfully quantifies inter‑patient variability - outperforming both nonlinear mixed‑effects baselines and recent neural ODE variants. Our code repository, pretrained model and tutorials are available online.

Code Dataset Promise: Yes

Code Dataset Url: https://github.com/cesarali/aicmet

Format Confirmation: I agree that I have read and followed the formatting instructions for the camera ready version.

Submission Number: 2414