Graph Gaussian Processes for Efficient Robust Monte Carlo Tree Search

Dorina Weichert, Samuel Wiest, Sebastian Houben, Paul Ploeger

Keywords: Reinforcement Learning, Monte Carlo Tree Search, Bayesian Optimization, Robustness, Graph Gaussian Processes

TL;DR: We derive a robust, sample-, and runtime-efficient algorithm integrating Graph Gaussian Processes into Monte Carlo Tree Search, outperforming the basic algorithms on simple robust and non-robust benchmarks.

Abstract: One of the major challenges in applying machine learning-based optimization algorithms in practice is their efficiency, which is measured in runtime and sample efficiency. Unfortunately, these two measures do not go hand in hand, but there are two poles: model-based Reinforcement Learning (RL), which is fast but requires many calls to some oracle, and Bayesian Optimization (BO), which has a high computation time but is very sample efficient. Additionally, both methods are at risk of oracle-misspecification: the oracle used for learning may differ from the final application. We derive Graph Gaussian Process Monte Carlo Tree Search (GUMTREES), an algorithm combining Monte Carlo Tree Search (MCTS), a model-based RL method, with BO, leading to an efficient algorithm that is easily enhanced to the robust setting. In a simple experiment, we demonstrate the superior performance of our algorithm.

Submission Number: 19