Executable Governance for AI: Translating Policies into Rules Using LLMs

AAAI 2026 Workshop AIGOV Submission7 Authors

Keywords: AI governance, Runtime policy enforcement, Policy-as-Code, Executable policies, Domain-specific language (DSL), Rule extraction, LLM-based policy mining, Compliance validation, Safety and robustness, Testable governance, Generative AI safeguards

TL;DR: We compile natural-language AI policies into executable tests via an LLM-driven DSL, enabling verifiable governance that measurably reduces violations.

Abstract:

AI policy guidance is predominantly written as prose, which practitioners must first convert into executable rules before frameworks can evaluate or enforce them. This manual step is slow, error-prone, difficult to scale, and often delays the use of safeguards in real-world deployments. To address this gap, we present Policy→Tests (P2T), a framework that converts natural-language policy documents into normalized, machine-readable rules. The framework comprises a pipeline and a compact domain-specific language (DSL) that en-
codes hazards, scope, conditions, exceptions, and required evidence, yielding a canonical representation of extracted rules. To test the framework beyond a single policy, we apply it across general frameworks, sector guidance, and enterprise standards, extracting obligation-bearing clauses and converting them into executable rules. These AI-generated rules closely match strong human baselines on span- and rule-level metrics, with robust inter-annotator agreement on the gold set. To evaluate downstream behavioral and safety im-
pact, we add HIPAA-derived safeguards to a generative agent and compare it with an otherwise identical agent without guardrails. An LLM-based judge, aligned with gold-standard criteria, measures violation rates and robustness to obfuscated and compositional prompts. Detailed results are provided in the appendix. We release the codebase, DSL, prompts, and rule sets as open-source resources to enable reproducible evaluation.

Submission Number: 7