Full-Stack Alignment: Co-Aligning AI and Institutions with Thicker Models of Value

Ryan Lowe, Joe Edelman, Tan Zhi-Xuan, Oliver Klingefjord, Ellie Hain, Vincent Wang, Atrisha Sarkar, Michiel A. Bakker, Fazl Barez, Matija Franklin, Andreas Haupt, Jobst Heitzig, Wesley H. Holliday, Julian Jara-Ettinger, Atoosa Kasirzadeh, Ryan Othniel Kearns, James Ravi Kirkpatrick, Andrew Koh, Joel Lehman, Sydney Levine et al. (2 additional authors not shown)

Keywords: sociotechnical alignment, human values, norms, human flourishing, markets, democratic institutions

Abstract:

AI alignment cannot be solved by focusing on a single system in isolation; even perfectly intent-aligned AI will lead to dangerous outcomes if embedded within institutions that are misaligned with human flourishing. We call this problem \Full-Stack Alignment (FSA): the co-alignment of AI systems and institutions with human agency and flourishing at all levels of society. We group current approaches to both AI alignment and institution design into two paradigms: one models values as any conceivable utility function or preference relation, as familiar from microeconomics, game theory, and mechanism design; another models values as any text string, prompt, or model-based critique. We argue that both paradigms struggle with problems like manipulation, value evolution, moral reasoning, and social context, making them ill-equipped to tackle the full scope of FSA.

Instead, we propose a new paradigm: thick models of value (TMV). Thick models of value impose structure on how we represent values and norms, so they can capture how individual well-being connects to collective good, distinguish genuine values from fleeting preferences, and embed individual choices within their social contexts. In other words, TMV takes a stand on what values and norms are, without imposing a singular vision of collective flourishing. TMV can apply to aligning both AI systems and institutions, making them a powerful tool for tackling Full-Stack Alignment; we make this argument using five key application areas (AI value stewardship, normatively competent agents, win-win negotiation systems, meaning-preserving economic mechanisms, and democratic regulatory institutions). Our aim is to articulate the conceptual foundations of TMV and nurture this emerging research into a coherent research program.

Submission Number: 77