Safe-CDT: Adaptive Target Scheduling for Safe Cross-Domain Deployment of Constrained Decision Transformers

Jiajun Shen, Bijan Sayyarrodsari

Keywords: safe reinforcement learning, decision transformer, offline-to-online deployment, constrained reinforcement learning, target-conditioned policies, deployment safety

TL;DR: Safe-CDT recalibrates RTG/CTG targets under environment shift, reporting a 14.4% violation rate on the main DSRL pair and environment-dependent safety behavior across additional pairs.

Abstract: Deploying Constrained Decision Transformers (CDT) across environments with different dynamics and safety budgets is challenging: fixed return-to-go and cost-to-go targets become brittle under distribution shift, leading to constraint violations. We propose Safe-CDT, a deployment framework combining adaptive target scheduling driven by Wilson-score achievability estimates, cost-aware multiplicative reweighting, and lightweight LoRA finetuning. We derive finite-episode probabilistic safety bounds, a critical-budget threshold, and a sufficient condition for cross-environment safety transfer. On the main DSRL cross-domain pair (CG1->CG2) under a consistent runtime-budget protocol, Safe-CDT achieves mean cost well below budget (8.56 at B=30) with the lowest observed violation rate among the baseline methods considered on that pair; additional pairs show environment-dependent safety behavior. We additionally evaluate CTG target responsiveness under environment shift via a 150-run target sweep, supporting the use of CTG conditioning as a deployment-time control variable for cost-reward tradeoff management.

Submission Number: 13