Using Deep Reinforcement Learning to Understand Odor Plume Tracking in Walking and Flying Agents

Aarav Sinha, Satpreet Harcharan Singh

Additional Submission Instructions: For the camera-ready version, please include the author names and affiliations, funding disclosures, and acknowledgements.

Track: Track 1: Original Research/Position/Education/Attention Track

Keywords: machine learning, deep reinforcement learning, recurrent neural network, neuroscience, biology

TL;DR: Using deep RL we show how walking and flying constraints lead to distinct navigation strategies and neural representations in odor plume tracking agents.

Abstract:

Odor plume tracking is critical for insect survival, yet how locomotion mode shapes navigation strategies remains unclear.
While the "cast-and-surge" behavior of flying insects is well studied, the strategies of walking insects have received less attention.
In this preliminary work, we use deep reinforcement learning to train biologically inspired agents in a physics-based environment, directly comparing walking and flying modes.
Walking agents, constrained by slower movement and limited turning, developed distinct strategies: fine-scale orientation relative to the plume centerline, pausing followed by localized search after plume loss, and subtle trajectory adjustments.
Flying agents instead relied on broad sweeping turns and rapid plume reacquisition.
Principal component analysis of recurrent activity revealed corresponding differences in neural representations: walking agents occupied compact, lower-dimensional manifolds, whereas flying agents exhibited continuous, higher-dimensional dynamics suited to flexible control.
Our work illustrates how reinforcement learning can generate normative models of insect navigation and demonstrate the utility of AI as a tool for uncovering general principles linking biomechanics, behavior, and neural computation.

Submission Number: 75