PointPatchRL - Masked Reconstruction Improves Reinforcement Learning on Point Clouds

Balazs Gyenes, Nikolai Franke, Philipp Becker, Gerhard Neumann

Keywords: Point Clouds, Self-Supervised Learning, Reinforcement Learning

TL;DR: Recent methods for representation learning on point clouds significantly improve RL on point clouds for challenging simulated robotic tasks

Abstract:

Perceiving the environment via cameras is crucial for Reinforcement Learning (RL) in robotics. While images are a convenient form of representation, they often complicate extracting important geometric details, especially with varying geometries or deformable objects. In contrast, point clouds naturally represent this geometry and easily integrate color and positional data from multiple camera views. However, while point-cloud processing with deep learning has seen many recent successes, RL on point clouds is under-researched, with only the simplest encoder architecture considered in the literature. We introduce PointPatchRL (PPRL), a method for RL on point clouds that builds on the common paradigm of dividing point clouds into overlapping patches, tokenizing them, and processing
the tokens with transformers. PPRL provides significant improvements compared with other point-cloud processing architectures previously used for RL. We then complement PPRL with masked reconstruction for representation learning and show that our method outperforms strong model-free and model-based baselines on image observations in complex manipulation tasks containing deformable objects and variations in target object geometry.

Website: https://alrhub.github.io/pprl-website

Code: https://github.com/balazsgyenes/pprl

Student Paper: no

Submission Number: 346