Scalable Online Planning via Reinforcement Learning Fine-TuningDownload PDF

21 May 2021, 20:47 (modified: 21 Jan 2022, 22:01)NeurIPS 2021 PosterReaders: Everyone
Keywords: Deep Reinforcement Learning, Search, Neural Network Fine-Tuning, Hanabi, Atari
TL;DR: By replacing tabular search with neural network fine-tuning via reinforcement learning, we achieve SOTA in Hanabi and we outperform MCTS in Ms. Pacman.
Abstract: Lookahead search has been a critical component of recent AI successes, such as in the games of chess, go, and poker. However, the search methods used in these games, and in many other settings, are tabular. Tabular search methods do not scale well with the size of the search space, and this problem is exacerbated by stochasticity and partial observability. In this work we replace tabular search with online model-based fine-tuning of a policy neural network via reinforcement learning, and show that this approach outperforms state-of-the-art search algorithms in benchmark settings. In particular, we use our search algorithm to achieve a new state-of-the-art result in self-play Hanabi, and show the generality of our algorithm by also showing that it outperforms tabular search in the Atari game Ms. Pacman.
Supplementary Material: zip
Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
13 Replies