Myriad: a real-world testbed to bridge trajectory optimization and deep learning

Nikolaus H. R. Howe, Simon Dufort-Labbé, Nitarshan Rajkumar, Pierre-Luc Bacon

Keywords: testbed, benchmark, real-world problems, reinforcement learning, imitation learning, optimal control, trajectory optimization, neural ordinary differential equations, system identification, end-to-end learning, implicit planning, nonlinear programming

Abstract: We present Myriad, a testbed written in JAX which enables machine learning researchers to benchmark imitation learning and reinforcement learning algorithms against trajectory optimization-based methods in real-world environments. Myriad contains 17 optimal control problems presented in continuous time which span medicine, ecology, epidemiology, and engineering. As such, Myriad strives to serve as a stepping stone towards application of modern machine learning techniques for impactful real-world tasks. The repository also provides machine learning practitioners access to trajectory optimization techniques, not only for standalone use, but also for integration within a typical automatic differentiation workflow. Indeed, the combination of classical control theory and deep learning in a fully GPU-compatible package unlocks potential for new algorithms to arise. We present one such novel approach for use in dynamics learning and control tasks. Trained in a fully end-to-end fashion, our model leverages an implicit planning module over neural ordinary differential equations, enabling simultaneous learning and planning with unknown environment dynamics. All environments, optimizers and tools are available in the software package at \url{https://github.com/nikihowe/myriad}.

Author Statement: Yes

URL: https://github.com/nikihowe/myriad

TL;DR: We present a testbed to benchmark imitation learning and reinforcement learning algorithms against trajectory optimization-based methods in real-world environments.

License: The repository is licensed under the Apache License, Version 2.0.

Contribution Process Agreement: Yes

In Person Attendance: Yes