Reinforcement Learning on a Futures Market Simulator

Koichi Moriyama, Mitsuhiro Matsumoto, Ken-ichi Fukui, Satoshi Kurihara, Masayuki Numao

Abstract: In recent years, it becomes vigorous to forecast a market by using machine learning methods. Since they assume that each trader’s individual decisions do not affect market prices at all, most existing works use a past market data set. Meanwhile there is an attempt to analyze economic phenomena by constructing a virtual market simulator, where human and artificial traders really make trades. Since prices in the market are determined by every trader’s decisions, it is more realistic and the assumption cannot be applied any more. In this work, we design and evaluate several reinforcement learners on a futures market simulator U-Mart (Unreal Market as an Artificial Research Testbed). After that, we compare our learner to the previous champions of U-Mart competitions.