Reflexivity in AI systems: metacognition, metalearning, self-improving systems, and second-guessing

ICLR 2026 Conference Submission21268 Authors

Keywords: Self-Attention Mechanisms, Metacognition, Meta-learning, Large Language Models, Self-Representation

TL;DR: Reflexivity is a central element of human intelligence and focusing research on it is key to bringing AI research to the next level.

Abstract: Our capacity for reflexivity (or self-representation) enables us to introspect on our thoughts and preferences, engage in metacognition, and evaluate our progress on projects while reconfiguring our approach as needed. Despite being central to our human intelligence, it has received marginal attention in AI research. To be clear, AI systems exist that manifest reflexivity. As we argue, meta-reinforcement learning modules and self-improving systems manifest reflexivity, as do Large Language Models that have the capacity for metacognition, so-called self-attention, and second-guessing. Reflexivity is the common denominator in each of these capacities and mechanisms. The manifestation of reflexivity in current AI systems is, however, a side-product of other goals. To date, no explicit attention has been directed at artificially replicating this central element of human intelligence. We propose that a general capacity for reflexivity is key to bringing AI to the next level.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 21268