Variable Impedance Control Combining Reinforcement Learning and Gaussian Process Regression

Paolino De Risi, Fabio Amadio, Gianluca Garofalo, Fanny Ficuciello, Pietro Falco

Abstract: Variable Impedance Control (VIC) approaches offer effective means for enabling robots to perform physical interaction tasks safely and proficiently, by including time-varying gains within an impedance control loop. However, determining the optimal gain profiles can be tedious and time-consuming. To address this challenge, this study introduces a VIC learning framework capable of autonomously acquiring suitable impedance behavior during task execution. This achievement is realized through the fusion of two techniques: (i) Reinforcement Learning (RL), to determine the most appropriate stiffness and damping gains for solving interaction tasks (e.g., lifting, pushing); and (ii) Gaussian Processes (GPs) for modeling and estimating optimal impedance parameters across task variations (e.g., changes in object weight). Consequently, we propose a Fast Cross-Entropy Method (FCEM) algorithm for autonomous stiffness learning, emphasizing all-the-time-stability to guarantee the stability of the control loop throughout the RL process. Additionally, we present a GP-based method to adapt impedance behaviors at run-time, adjusting stiffness based on online external torques estimates provided by a momentum observer (without requiring a wrench sensor). Experimental results on a simulated ABB Mobile YuMi robot show the framework’s capabilities across different tasks.