KeyCLD: Learning constrained Lagrangian dynamics in keypoint coordinates from images

Rembert Daems, Jeroen Taets, Francis Wyffels, Guillaume Crevecoeur

Abstract: Highlights•We learn constrained Lagrangian dynamics from images.•Learned keypoint representations from images are used to learn Lagrangian dynamics.•We perform control in Cartesian coordinates with energy shaping.•Empirical validation on the pendulum, cartpole and acrobot systems from cl_bnmsqnk.•KeyCLD achieves the highest valid prediction time on all benchmarks.