Triple Regression for Sim2Real Adaptation in Human-Centered Robot Grasping and Manipulation

Yuanhong Zeng, Yizhou Zhao, Ying Nian Wu

Keywords: Sim2Real, Robot, Manipulation

Abstract: Sim2Real (Simulation to Reality) techniques have gained prominence in advancing robot grasping and manipulation, particularly in human-centered environments where adaptation to dynamic, unpredictable conditions is essential. This paper introduces the Triple Regression framework, which uses a digital twin in real time to enhance robot learning and interaction. The framework addresses the reality gap in human-robot collaboration by: (1) mitigating projection errors between real and simulated camera perspectives through dual regression models and (2) detecting and compensating for discrepancies in robot control using a third regression model. Experiments in picking up and pouring water from random places, which are critical for collaborative robots in dynamic human environments, demonstrate the effectiveness of our method with only the input of an RGB camera. Our work enhances adaptive, real-time robotic decision making in collaborative human-centered tasks and pushes the boundaries of robot learning in both simulation and real-world scenarios.

Submission Number: 9