DRL Trading with CPT Actor and Truncated Quantile Critics

Marcus Jun Rong Foo, Nixie S. Lesmana, Chi Seng Pun

Abstract: The Cumulative Prospect Theory (CPT) is a popular behavioral decision-making model that has been shown to reflect humans’ risk-sensitive behavior. This work develops an end-to-end CPT-based DRL trading agent. For our architecture, we draw on the Truncated Quantile Critics (TQC), an actor-critic distributional RL method designed to learn return distributions for risk-neutral continuous control while guarding against overestimation bias. We introduce a CPT actor and realistic trading constraints to TQC to build a novel TQ2CPT trading algorithm. We evaluate the performance of our algorithm against several benchmark strategies in various portfolio metrics and demonstrate CPT’s efficacy as a risk measure for DRL trading, as well as its synergistic relationships with TQC.