Playful and Exploratory Behavior from the Maximum Occupancy Principle

Published: 09 Oct 2024, Last Modified: 02 Dec 2024NeurIPS 2024 Workshop IMOL PosterEveryoneRevisionsBibTeXCC BY 4.0
Track: Full track
Keywords: Intrinsic Motivation, Reinforcement Learning, Curiosity, Behavioural Variability, Neural Variability
TL;DR: Curiosity and exploration can emerge from the intrinsic motivation to occupy action space.
Abstract: We build on the Maximum Occupancy Principle (MOP) and show complex and playful behavior emerging from intrinsic motivation to occupy action space. Relevantly, the drive to occupy action space as uniformly as possible in the long run, while avoiding terminal states, leads to interesting behaviors, such as non-trivial interaction with external objects. We show that MOP agents in navigation tasks are inherently curious, as they are attracted by the possibility of playing with available objects or using them as tools to visit larger regions of space. This principle is then extended to neural activity (NeuroMOP). We introduce a more complex continuous navigation problem where the motor output of the agent is defined by two units of a recurrent neural network of fixed weights. We show that a MOP controller can drive the network's activity and lead the motor output units to occupy the whole available space. This example highlights the potential of MOP as a principle not only for behavior but also for neural activity. All together, these results indicate MOP as a possible principle underlying various aspects of natural behavior, reconciling multiple perspectives of intrinsic motivation, such as curiosity and exploration.
Submission Number: 10
Loading