Declare and Justify: Explicit assumptions in AI evaluations are necessary for effective regulation

Peter Barnett, Lisa Thiergart

Keywords: Evaluations, regulation, risk management

TL;DR: We identify key assumptions in using evaluations to prevent harm from AI systems, and propose that regulation should require developers to adequately justify these assumptions.

Abstract: As AI systems advance, AI evaluations are becoming an important pillar of regulations for ensuring safety. We argue that such regulation should require developers to explicitly identify and justify key underlying assumptions about evaluations as part of their case for safety. We identify core assumptions in AI evaluations (both for evaluating existing models and forecasting future models), such as comprehensive threat modeling, proxy task validity, and adequate capability elicitation. Many of these assumptions cannot currently be well justified. If regulation is to be based on evaluations, it should require that AI development be halted if evaluations demonstrate unacceptable danger or if these assumptions are inadequately justified. Our presented approach aims to enhance transparency in AI development, offering a practical path towards more effective governance of advanced AI systems.

Submission Number: 52