Robust Locomotion Policy With Adaptive Lipschitz Constraint for Legged Robots

Yang Zhang, Buqing Nie, Yue Gao

Abstract: Deep Reinforcement Learning (DRL) has achieved significant advancements in legged robot locomotion tasks. However, neural network-based control policies suffer from action fluctuations in complex environments due to sensor noise and external disturbances, resulting in decreased locomotion performance. In this work, we propose a novel robust locomotion policy schema leveraging Multi-dimensional Gradient Normalization (MGN) to achieve policy parameterization that satisfies Lipschitz constraint at the network level, and constructs a Lipschitz Constant Network (LCN) to dynamically adjust the local Lipschitz constant of the motion policy to balance motion performance and robustness. The proposed Lipschitz Locomotion Policy (LLP) improves the smoothness of actions and is robust to observation noise and external disturbances. We validate our method in both simulation and real deployment, demonstrating that it outperforms existing methods in velocity tracking performance and the success rate of traversing complex terrain.