PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets

Shuo Sun, Molei Qin, Xinrun Wang, Bo An

Abstract: The financial markets, which involve more than $90 trillion market capitals, attract the attention of innumerable investors around the world. Recently, reinforcement learning in financial markets (FinRL) has emerged as a promising direction to train agents for making profitable investment decisions. However, the evaluation of most FinRL methods only focuses on profit-related measures and ignores many critical axes, which are far from satisfactory for financial practitioners to deploy these methods into real-world financial markets. Therefore, we introduce PRUDEX-Compass, which has 6 axes, i.e., Profitability, Risk-control, Universality, Diversity, rEliability, and eXplainability, with a total of 17 measures for a systematic evaluation. Specifically, i) we propose AlphaMix+ as a strong FinRL baseline, which leverages mixture-of-experts (MoE) and risk-sensitive approaches to make diversified risk-aware investment decisions, ii) we evaluate 8 FinRL methods in 4 long-term real-world datasets of influential financial markets to demonstrate the usage of our PRUDEX-Compass, iii) PRUDEX-Compass together with 4 real-world datasets, standard implementation of 8 FinRL methods and a portfolio management environment is released as public resources to facilitate the design and comparison of new FinRL methods. We hope that PRUDEX-Compass can not only shed light on future FinRL research to prevent untrustworthy results from stagnating FinRL into successful industry deployment but also provide a new challenging algorithm evaluation scenario for the reinforcement learning (RL) community.

Certifications: Reproducibility Certification

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

Here are a summary of changes since last submission:
* We include discussion on the potential impact of FinRL community as AE suggested.
* We reorganise and polish the Appendix part to make it more clear.
* We proofread the paper and fix typos.
* We include more comprehensive and accurate citations on related works.

Code: https://github.com/TradeMaster-NTU/PRUDEX-Compass

Assigned Action Editor: ~Wei_Liu3

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 598