Do evaluation functions really improve Monte-Carlo tree search?

Kiminori Matsuzaki, Naoki Kitamura

Abstract: Monte-Carlo tree search (MCTS) algorithms play an important role in developing computer players, especially for games for which good evaluation functions are hard to obtain, like Go. The performance of MCTS players is often leveraged in combination with online and/or offline knowledge, despite the lack of game-theoretic guarantees. For the games for which we already have good evaluation functions, the use of evaluation functions in MCTS algorithms achieved a success. However, the effect of evaluation functions on the performance of MCTS algorithms have not been investigated well, especially in terms of the quality of evaluation functions. In this study, we try to address this issue by using Othello (Reversi) as the target game. Based on the evaluation function used in Zebra, a top-level open-source player, we design 15 variants of evaluation functions and use them in three ways in MCTS algorithms. We conduct a set of experiments exhaustively and analyze the effect of evaluation functions in MCTS algorithms.