Abstract: This paper presents Park, an open extensible platform that uses a common interface to connect to a suite of real world computer systems for RL augmented optimizations. These systems cover a wide spectrum of problems, including both global vs. distributed control, and fast control loop vs. long term planning. This dataset unveils unique challenges that the existing off-the-shelf RL techniques cannot solve. The challenges occur in the representation and search of the state-action space, the special property of the decision process and the reality gap between simulations and actual systems. To understand the effect of these challenges, we benchmark several existing RL algorithms in Park with comparing heuristic baselines.