Marginal Risk Relative to What? Distinguishing Baselines in AI Risk Management

Jide Alaga, Michael Chen

Keywords: Machine Learning, Artificial Intelligence Safety, AI Governance, Risk Assessment, Foundation Models, Ethics in AI, Technology Policy, Responsible AI, Model Evaluation, Frontier AI, ICML

TL;DR: This paper examines two approaches to AI risk assessment: comparing models to a world before general-purpose AI versus comparing their risk to existing AI systems, and recommends greater transparency about which baseline is being used.

Abstract: Major developers of large foundation models make development and deployment decisions informed by evaluations of "marginal risk": risk introduced by a new AI model, relative to a baseline. Developers face a critical choice between two types of baselines: a "pre-GPAI" baseline without modern general-purpose AI systems (e.g., only having 2023-level technology), or a "post-GPAI" baseline which includes the most risk-enabling models already available. Reviewing voluntary safety frameworks adopted by AI model developers, we note that developers do not always clearly specify which baseline is used. We examine potential risks of cumulative model releases that incrementally add marginal risk, leading to an environment in which each individual model may appear safe from the perspective of post-GPAI baselines, while aggregate risk from AI becomes unacceptably dangerous.

Submission Number: 65