GRL-SNAM: Geometric Reinforcement Learning with Differential Hamiltonians for Navigation and Mapping in Unknown Environments

Aditya Sai Ellendula, Yi Wang, Minh Phuong Nguyen, Chandrajit L. Bajaj

Keywords: Reinforcement Learning, Generalized Hamiltonian Manifolds, Differential Policy Optimization

TL;DR: We propose a geometric RL method that navigates and maps using only local sensing, leveraging Hamiltonian dynamics and differential policy optimization to adapt quickly under dynamic, deformable conditions

Abstract: We present GRL-SNAM, a geometric reinforcement learning framework for Simultaneous Navigation and Mapping in unknown environments. GRL-SNAM differs from traditional SLAM and other reinforcement learning methods by relying exclusively on local sensory observations without constructing a global map. Our approach formulates navigation and mapping as coupled dynamics on generalized Hamiltonian manifolds: sensory inputs are translated into local energy landscapes that encode reachability, obstacle barriers, and deformation constraints, while policies for sensing, planning, and reconfiguration evolve stagewise under Differential Policy Optimization. A reduced Hamiltonian serves as an adaptive score function, updating kinetic/potential terms, embedding barrier constraints, and continuously refining trajectories as new local information arrives. We evaluate GRL-SNAM on 2D deformable navigation tasks, where a hyperelastic robot learns to squeeze through narrow gaps, detour around obstacles, and generalize to unseen environments. We compare our method against local reactive baselines (PF, CBF, staged DWA), global A* references (rigid, clearance-aware), and deep RL baselines (PPO, TRPO, SAC) under identical stagewise sensing constraints. GRL-SNAM achieves strong path quality while using the minimal map coverage, preserves clearance, and generalizes to unseen layouts. This demonstrates that our Hamiltonian-structured RL framework enables high-quality navigation through minimal exploration via local energy refinement rather than global mapping.

Primary Area: reinforcement learning

Submission Number: 19476