Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols

Charlie Griffin, Louis Thomson, Buck Shlegeris, Alessandro Abate

Keywords: Partially Observable Stochastic Games, AI Control, AI Evaluations, Safeguards, Game theory

TL;DR: We introduce a game-theoretic framework for modelling AI Control evaluations, and synthesising protocols.

Abstract:

To evaluate the safety and usefulness of deployment protocols for untrusted AIs, AI Control uses a red-teaming exercise played between a protocol designer and an adversary. This paper introduces AI-Control Games, a formal decision-making model of the red-teaming exercise as a multi-objective, partially observable, stochastic game. We also introduce reductions from AI-Control Games to a special case of zero-sum partially observable stochastic games that allow us to leverage existing algorithms to find Pareto-optimal protocols.
We apply our formalism to model, evaluate and synthesise protocols for deploying untrusted language models as programming assistants, focusing on Trusted Monitoring protocols, which use weaker language models and limited human assistance.
Finally, we demonstrate the utility of our formalism by showcasing improvements over empirical studies in existing settings, evaluating protocols in new settings, and analysing how modelling assumptions affect the safety and usefulness of protocols.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 25452