Standard Market Environments for Financial Reinforcement Learning

Chunlin Feng, Lijian Huang, Keyi Wang, Yupeng Cao, Ming Zhu, Jiechao Gao, Xiao-Yang Liu

Keywords: Financial reinforcement learning, market environments, FinRL, stock trading, crypto trading, standard evaluation pipeline.

TL;DR: This paper proposes standard evaluation pipelines for stock and crypto market environments, and evaluate four open-source market environments.

Abstract: Market environments play a critical role in obtaining a robust trading agent. However, current open-source market environments are unorganized, hindering reproduction in the community. In this paper, we organize four market environments. First, we introduce two standard evaluation pipelines, corresponding to stock and crypto markets. Each pipeline integrates financial data, performance metrics, and baseline strategies. Within these pipelines, two stock market environments and two crypto market environments are evaluated. They are compared with the mean-variance porfolio allocation strategy, equal-weight strategy, and market indices, DJIA (stock) and S&P BDA (crypto). The environments are standardized to Gymnasium-Style with documentation provided. Both stock environments outperform DJIA and equal-weight strategy in annual return, and both crypto environments outperform all three baselines in annual return.

Submission Number: 103