Deep reinforcement learning with dynamic graphs for adaptive informative path planning

Apoorva Vashisth, Julius Rückin, Federico Magistri, Cyrill Stachniss, Marija Popovic

Abstract:

Autonomous robots are often employed for data
collection due to their efficiency and low labour costs. A key
task in robotic data acquisition is planning paths through an
initially unknown environment to collect observations given
platform-specific resource constraints, such as limited battery
life. Adaptive online path planning in 3D environments is
challenging due to the large set of valid actions and the presence
of unknown occlusions. To address these issues, we propose
a novel deep reinforcement learning approach for adaptively
replanning robot paths to map targets of interest in unknown
3D environments. A key aspect of our approach is a dynamically
constructed graph that restricts planning actions local to the
robot, allowing us to react to newly discovered static obstacles
and targets of interest. For replanning, we propose a new
reward function that balances between exploring the unknown
environment and exploiting online-discovered targets of interest.
Our experiments show that our method enables more efficient
target discovery compared to state-of-the-art learning and non-
learning baselines. We also showcase our approach for orchard
monitoring using an unmanned aerial vehicle in a photorealistic
simulator. We open-source our code and model at: https:
//github.com/dmar-bonn/ipp-rl-3d.