Adaptive patch foraging in deep reinforcement learning agents

Nathan Wispinski, Andrew Butcher, Kory Wallace Mathewson, Craig S Chapman, Matthew Botvinick, Patrick M. Pilarski

Abstract: Patch foraging is one of the most heavily studied behavioral optimization challenges in biology. However, despite its importance to biological intelligence, this behavioral optimization problem is understudied in artificial intelligence research. Patch foraging is especially amenable to study given that it has a known optimal solution, which may be difficult to discover given current techniques in deep reinforcement learning. Here, we investigate deep reinforcement learning agents in an ecological patch foraging task. For the first time, we show that machine learning agents can learn to patch forage adaptively in patterns similar to biological foragers, and approach optimal patch foraging behavior when accounting for temporal discounting. Finally, we show emergent internal dynamics in these agents that resemble single-cell recordings from foraging non-human primates, which complements experimental and theoretical work on the neural mechanisms of biological foraging. This work suggests that agents interacting in complex environments with ecologically valid pressures arrive at common solutions, suggesting the emergence of foundational computations behind adaptive, intelligent behavior in both biological and artificial agents.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: Paper accepted. Deanonymized authors, affiliations, and acknowledgements. Changed to TMLR accepted version of Latex template.

Video: https://youtu.be/5PfxPj5Jzwo

Assigned Action Editor: ~Blake_Aaron_Richards1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 667