Bitter Lesson of the ARC-AGI Challenge: Intelligence may look very different in machines and humans

Soumya Banerjee

Keywords: anthropocentrism

Abstract: The Abstraction and Reasoning Corpus (ARC) and the associated ARC-AGI challenge are benchmarks for evaluating core reasoning skills. Recently, OpenAI's new model o3 solved ARC-AGI, reigniting debate on whether such achievements reflect true reasoning or mere pattern matching. Rich Sutton's "Bitter Lesson" suggests that general methods at scale outperform specialized ones; in ARC, top solutions heavily rely on data augmentation. We examine this tension and propose that new concepts or metaphors may be needed to describe machine reasoning. This position paper argues that reasoning and intelligence in machines can differ fundamentally from human cognition: recognizing that intelligence is task-specific and diverse across animals, humans, and machines could lead to more appropriate benchmarks.

Submission Number: 3