The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?

ICLR 2026 Conference Submission24103 Authors

Keywords: Large Language Models, Scaling Laws, Misalignment, Bias-Variance

TL;DR: We propose decomposing AI error with the bias and variance framework, and analyze scaling behavior across task complexity and model scale.

Abstract:

As AI becomes more capable, we entrust it with more general and consequential tasks. The risks from failure grow more severe with increasing task scope. It is therefore important to understand
the ways extremely capable AI models will fail: Will they fail by systematically pursuing goals we do not intend? Or will they fail by being a hot mess, and taking nonsensical actions that do not further any goal? We operationalize this question using a bias-variance decomposition of the errors made by AI models: An AI's *incoherence* on a task is measured over test-time randomness as the fraction of its error that stems from variance rather than bias in task outcome. Across all tasks and frontier models we measure, we find that the longer models spend reasoning and taking actions, *the more incoherent* they become. We observe that incoherence changes with model scale in a way that is task and experiment dependent. However, in several settings larger, more capable models are more incoherent than smaller models.
Consequently, scale alone seems unlikely to eliminate incoherence. Instead, as more capable AIs pursue harder tasks, requiring more sequential action and thought, our results predict failures to be accompanied by more incoherent behavior.
This suggests a future where AIs sometimes cause industrial accidents (due to unpredictable misbehavior), but are less likely to exhibit consistent pursuit of a misaligned goal.
This increases the relative importance of alignment research targeting reward hacking or goal misspecification.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 24103