The Controllability Trap: A Governance Framework for Military AI Agents

Subramanyam Sahoo

Keywords: Agentic AI, military AI governance, Provenance Tracking, Corrigibility

TL;DR: We propose AMAGF, a three-pillar governance framework with a real-time Control Quality Score (CQS) to measure, detect, and correct human control failures across six agentic risk modes in military multi-agent AI systems.

Abstract:

Agentic AI systems—capable of goal interpretation, world modeling, planning, tool use, long-horizon operation, and autonomous coordination—introduce distinct control failures not addressed by existing safety frameworks. We identify six agentic governance failures tied to these capabilities and show how they erode meaningful human control in military settings.
We propose the Agentic Military AI Governance Framework (AMAGF), a measurable architecture structured around three pillars: Preventive Governance (reducing failure likelihood), Detective Governance (real-time detection of control degradation), and Corrective Governance (restoring or safely degrading operations). Its core mechanism, the Control Quality Score (CQS), is a composite real-time metric quantifying human control and enabling graduated responses as control weakens. For each failure type, we define concrete mechanisms, assign responsibilities across five institutional actors, and formalize evaluation metrics. A worked operational scenario illustrates implementation, and we situate the framework within established agent safety literature. We argue that governance must move from a binary conception of control to a continuous model in which control quality is actively measured and managed throughout the operational lifecycle.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 245