Open Problems in Technical AI Governance

Anka Reuel, Benjamin Bucknall, Stephen Casper, Timothy Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Sasha Luccioni et al. (13 additional authors not shown)

Abstract: AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, outline why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

Certifications: Survey Certification

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

We addressed reviewer feedback as outlined in our rebuttal comments. Specifically, we have:

- Added a detailed methodological description clarifying how the taxonomy and open problems were identified.
- Enhanced the clarity and specificity of the definition of Technical AI Governance (TAIG), for example, by explicitly defining the term 'technical.'
- Clarified distinctions between TAIG and related fields, and expanded the introduction and related literature discussions to better contextualize TAIG within the broader field and literature.
- Differentiated the benefits of TAIG as a field from the specific contributions of our paper.
- Improved justification and provided additional context on why problematic data identification is valuable, particularly in scenarios where direct dataset access is not possible.
- Tightened definitions and clarified distinctions among key terms throughout the paper.
- Significantly expanded the conclusion section to provide a comprehensive synthesis of insights and contributions from Sections 3–8.
Included additional relevant references throughout the paper.

These improvements directly address the reviewers' concerns. Further details are available in the reviewers' comments and our corresponding rebuttals.

We sincerely thank all reviewers and the Action Editor for thoroughly engaging with our paper and providing thoughtful feedback, which substantially enhanced the quality of our work.

Thank you!

Assigned Action Editor: ~Stefan_Lee1

Submission Number: 3552