Towards A New Toolkit for Measuring AI-Enabled Influence Operations

Shannon Yang

Keywords: influence operations, evaluations, ai forecasting, dangerous capabilities uplift, criminal misuse

Abstract:

Recent advancements in AI technologies further reinforce the need for a new framework for measuring AI-enabled influence operations (IOs), defined as efforts to influence public or political debate and decision-making processes that rely in part or in whole on covert activity. In 2020, Brookings published the Breakout Scale by Ben Nimmo, which categorizes influence operations based on the number of platforms influenced, breakouts into mainstream media, and policy response or a call to violence. While comprehensive and appropriate with regard to one prominent type of IOs (those seeking virality to achieve their goals), the Breakout Scale showed its limits with operations where wide exposure of and engagement with the posted content was not necessary.

In April 2025, Anthropic reported on several malicious uses of Claude that they detected and countered, including an AI-enabled influence-as-a-service operation utilizing fake social media personas to promote political narratives on behalf of clients. Anthropic noted that: “These operations suggest a need for new frameworks
for evaluating influence operations centered around relationship building and community integration, in addition to the existing Breakout Scale which focuses
on viral impact or breakout moments. While we have disrupted this specific operation, we expect this model to become increasingly common as AI lowers the barrier to entry for sophisticated influence campaigns.” AI allows malicious actors to expand their capabilities across the board, meaning that the delta between the current impact measurement frameworks – including the Breakout Scale – and the nature and tactics of IOs will likely increase in size.

In this paper, we put forth a new framework that takes into account the limitations of the Breakout Scale when it comes to a wider range of influence operations, including those which utilize AI: accounting for individual human-AI interactions, community integration, and information inherent to models. We look at specific case studies in which the Breakout Scale’s effectiveness in analyzing the impact of an operation is limited, but the new framework is able to accurately measure it.

In order to ensure that the transition to advanced AI goes well, we must envision a world in which humans are empowered to distill information accurately. AI-enabled influence operations undermine this vision. By developing a broader understanding of influence operations that looks beyond virality alone, we can 1) more accurately measure the impact of a wider range of operations, 2) assess any uplift resulting from the use of AI, and 3) develop best practices to quantify risk and harm from AI-enabled influence operations and take steps toward best practices to reduce them.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 42