Gentle Manipulation of Tree Branches: A Contact-Aware Policy Learning Approach

Jay Jacob, Shizhe Cai, Paulo Vinicius Koerich Borges, Tirthankar Bandyopadhyay, Fabio Ramos

Keywords: Reinforcement Learning, Sim-to-Real, Deformable Manipulation

TL;DR: A reinforcement learning approach using a contact detection classifier and sim-to-real transfer for gentle interaction with tree branches.

Abstract: Learning to interact with deformable tree branches with minimal damage is challenging due to their intricate geometry and inscrutable dynamics. Furthermore, traditional vision-based modelling systems suffer from implicit occlusions in dense foliage, severely changing lighting conditions, and limited field of view, in addition to having a significant computational burden preventing real-time deployment.In this work, we simulate a procedural forest with realistic, self-similar branching structures derived from a parametric L-system model, actuated with crude spring abstractions, mirroring real-world variations with domain randomisation over the morphological and dynamic attributes. We then train a novel Proprioceptive Contact-Aware Policy (PCAP) for a reach task using reinforcement learning, aided by a whole-arm contact detection classifier and reward engineering, without external vision, tactile, or torque sensing. The agent deploys novel strategies to evade and mitigate contact impact, favouring a reactive exploration of the task space. Finally, we demonstrate that the learned behavioural patterns can be transferred zero-shot from simulation to real, allowing the arm to navigate around real branches with unseen topology and variable occlusions while minimising the contact forces and expected ruptures.

Website: https://sites.google.com/view/pcap/home

Student Paper: yes

Submission Number: 478