Selection of Exploratory or Goal-Directed Behavior by a Physical Robot Implementing Deep Active Inference

Ko Igari, Kentaro Fujii, Gabriel W. Haddon-Hill, Shingo Murata

Abstract: Intelligent robots are being developed with the expectation that they will perform various tasks in diverse environments. Such robots need to autonomously engage in both exploratory behavior to reduce environmental uncertainty and goal-directed behavior to achieve their preferred observations (or goals). In this study, we focus on active inference, which provides a unified scheme for these distinct behavioral modes. Policy selection in active inference is based on minimizing expected free energy (EFE), which consists of one term representing epistemic value and another representing extrinsic value. Specifically, we investigate the influence of preference precision, which controls the balance between these two terms, on policy selection by a physical robot receiving high-dimensional and uncertain observations. We developed a deep active inference framework comprising a world model and a policy suggester. The world model predicts future hidden states and observations based on candidate policies from the policy suggester. The EFE for each policy is approximated using the predicted future hidden states and observations as well as the preferred observation. We implemented our proposed framework in a robot, requiring it to select a policy that minimizes EFE and then generate actions accordingly. The experimental results showed that the robot implementing the proposed framework selected exploratory or goal-directed behavior depending on the level of preference precision. These findings suggest that adjusting preference precision plays a crucial role in the autonomous selection of exploratory or goal-directed behavior in real-world situations with potential uncertainty.

External IDs:dblp:conf/iwai2/IgariFHM24