High-Fidelity Simulation of a Cartpole for Sim-to-Real Deep Reinforcement Learning

Linus Bantel, Peter Domanski, Dirk Pflüger

Abstract: This work proposes a novel physics-based Cartpole simulation environment as a new benchmark to address the sim-to-real transfer. Our simulation environment extends the original Gymnasium Cartpole environment with additional physics and data-driven models for friction, air resistance, and the nonlinear behavior of the applied force on the cart inspired by a real-world experimental setup. We implement the Gymnasium environment interface, allowing us to use our implementation as a drop-in replacement with configurable simulation fidelity. We show that our physics-based Cartpole simulation with Reinforcement Learning minimizes the reality gap to our real-world Cartpole setup without increasing computational efforts considerably. Moreover, our simulation environment is an efficient surrogate model for a real Cartpole, and thus provides a rare example of closing the reality gap.