PufferLib 2.0: Reinforcement Learning at 1M steps/s

Joseph Suarez

Keywords: PufferLib, Reinforcement Learning, Library, Tools

TL;DR: PufferLib is a high-performance RL library with a suite of environments that run at 1M steps/second

Abstract:

PufferLib is an open-source reinforcement learning project built around efficient and
broadly compatible simulation. Our first-party suite of 12 environments each run at 1M
steps/second. For existing environments, PufferLib provides one-line wrappers that eliminate
common compatibility problems and fast vectorization to accelerate training. With PufferLib,
you can use familiar libraries like CleanRL and SB3 to scale from classic benchmarks like
Atari and Procgen to complex simulators like NetHack and Neural MMO 3.

Submission Number: 151