Cascading Effects: A Multifaceted Governance Challenge in AI Systems

Anna Neumann, Jat Singh

Keywords: AI Supply Chains, Governance, Private Governance, Regulation, AI, LLMs, Cascade, Audit, Evaluation

TL;DR: AI systems combine components that behave differently when connected than isolated, creating cascading effects that challenge traditional evaluations and require new governance approaches accounting for multifaceted challenges.

Abstract:

AI-based systems entail combinations of foundation models and other mechanisms, like tools, memory systems, and surrounding prompt scaffolding. This can be described as a
chain of interconnected different components, for example AI agents and multi-agent workflows.
While these chains commodify building AI applications and systems, they present a key challenge: individual components may exhibit different behaviours when connected to each other than when separated, affecting the properties of the overall system. This means that isolated evaluations and audits of each part do not ensure a safe and reliable overall system.

This paper describes how effects of components interacting can cascade throughout systems and result in evaluation challenges, and a discusses on benefits of cascade-level analyses for (private) governance.

Submission Number: 18