Twin-Delayed Deep Deterministic Policy Gradient Algorithm for Portfolio Selection

Nicholas Baard, Terence L. van Zyl

Abstract: State-of-the-art RL algorithms have shown suboptimal performance in some market conditions with regard to the portfolio selection problem. The reason for suboptimal performance could be due to overestimation bias in actor-critic methods through the use of neural networks as the function approximator. The resulting bias leads to a suboptimal policy being learned by the agent, hindering performance. This research focuses on using the Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm for portfolio selection to achieve greater results than previously achieved. In addition, an analysis of the overall effectiveness of the algorithm in various market conditions is needed to determine the TD3’s robustness. This research establishes a RL environment for portfolio selection and trains the TD3 alongside three state-of-the-art algorithms in five different market conditions. The algorithms are tested by allowing the agent to manage a portfolio in each market for a specified period. The results are used for the analysis of the algorithms. The research shows improved results achieved by the TD3 algorithm for portfolio selection compared to other state-of-the-art algorithms. Furthermore, the performance of the TD3 across the five selected markets proves the robustness of the algorithm in its use for the portfolio selection problem.