Of Mice and Machines: A Comparison of Learning Between Real World Mice and RL Agents

Shuo Han, German Espinosa, Junda Huang, Daniel Dombeck, Malcolm A. MacIver, Bradly C. Stadie

TL;DR: We identify key differences between mice and RL agents in predator-avoidance tasks and develop new methods to make artificial agents behave more cautiously like their biological counterparts.

Abstract: Recent advances in reinforcement learning (RL) have demonstrated impressive capabilities in complex decision-making tasks. This progress raises a natural question: how do these artificial systems compare to biological agents, which have been shaped by millions of years of evolution? To help answer this question, we undertake a comparative study of biological mice and RL agents in a predator-avoidance maze environment. Through this analysis, we identify a striking disparity: RL agents consistently demonstrate a lack of self-preservation instinct, readily risking ``death'' for marginal efficiency gains. These risk-taking strategies are in contrast to biological agents, which exhibit sophisticated risk-assessment and avoidance behaviors. Towards bridging this gap between the biological and artificial, we propose two novel mechanisms that encourage more naturalistic risk-avoidance behaviors in RL agents. Our approach leads to the emergence of naturalistic behaviors, including strategic environment assessment, cautious path planning, and predator avoidance patterns that closely mirror those observed in biological systems.

Lay Summary:

Imagine you're playing a game where you need to reach your goal while avoiding a dangerous robot. How would you play compared to a real mouse in the same situation? This research reveals a fascinating difference: while computer programs (artificial intelligence) often take risky shortcuts to complete tasks faster, real animals like mice are much more cautious and safety-focused.

This highlights a crucial gap between artificial and biological intelligence. AI systems excel at optimizing for specific objectives but lack the self-preservation instincts that evolution has built into living creatures.

Although our experiment focuses on a specific scenario, predator-prey interactions are well-studied and complex situations in biology. We developed two new techniques that help AI agents learn to be more cautious, leading our AI agents to behave much more like real mice—taking safer paths and spending more time assessing their environment.

This work contributes to understanding mouse behavior and bridging the gap between artificial and biological decision-making.

Primary Area: Reinforcement Learning->Everything Else

Keywords: Reinforcement learning, Biological agents, Comparative study, Variance-penalized temporal difference learning, Experience replay, Behavioral alignment

Submission Number: 8136