Predictive Minds: LLMs As Atypical Active Inference Agents
Keywords: Large Language Models, Active Inference, Predictive Processing, Generative Models, Emergent Control Loops, Societal Ramifications of AI
TL;DR: Interpreting LLMs in the active inference frame leads us to many similarities, and make a prediction about enhanced self-awareness and agency if the feedback between actions and perceptions gets tighter
Abstract: Large language models (LLMs) like GPT are often conceptualized as passive predictors, simulators, or even 'stochastic parrots'. We instead conceptualize LLMs by drawing on the theory of active inference originating in cognitive science and neuroscience. We examine similarities and differences between traditional active inference systems and LLMs, leading to the conclusion that, currently, LLMs lack a tight feedback loop between acting in the world and perceiving the impacts of their actions, but otherwise fit in the active inference paradigm. We list reasons why this loop may soon be closed, and possible consequences of this including enhanced model self-awareness and the drive to minimize prediction error by changing the world.
Submission Number: 118
Loading