Feasibility‑Guided Fair Adaptive Reinforcement Learning for Medicaid Care Management

Sanjay Basu

Keywords: offline reinforcement learning, healthcare, fairness, safety, adaptive policy switching, diffusion models, Medicaid

TL;DR: We introduce FCAF‑RL, a safe and fair offline reinforcement‑learning framework that combines diffusion‑based augmentation, equalised‑odds regularisation and adaptive policy switching to reduce acute events and disparities in Medicaid care management.

Abstract: Care‑management programmes for Medicaid populations must balance reductions in acute events with equitable treatment across demographic groups. Existing reinforcement‑learning methods either ignore fairness or rely on online exploration, raising safety concerns. We propose Feasibility‑Guided Fair Adaptive Reinforcement Learning (FCAF‑RL), an offline framework that unifies three advances—diffusion‑based safety augmentation, equalised‑odds fairness regularisation and adaptive policy switching—to learn safe and fair intervention policies from retrospective data. Using weekly trajectories of 155 631 Medicaid beneficiaries across Washington, Virginia and Ohio (Jan 2023–Jun 2024), we model care management as a partially observable Markov decision process with nine possible interventions. A diffusion model augments logged data within a clinician‑defined feasible region; multiple Q‑networks are trained with varying fairness weights using a conservative Bellman objective; and a deployment rule selects among these policies based on realised disparities. In leave‑one‑state‑out evaluation, FCAF‑RL reduced acute events by 31 % relative to a risk‑based baseline and 21 % relative to Implicit Q‑Learning, while lowering fairness disparities from 8.9 to 2.5 percentage points. These results suggest that integrating safety, fairness and adaptability can meaningfully improve care management equity without online experimentation. Code and a synthetic dataset will be released upon acceptance to facilitate reproducibility.

Submission Number: 40