Learning Heuristics for Automated Reasoning through Reinforcement Learning

Gil Lederman, Markus N. Rabe, Edward A. Lee, Sanjit A. Seshia

Abstract: We demonstrate how to learn efficient heuristics for automated reasoning algorithms through deep reinforcement learning. We focus on backtracking search algorithms for quantified Boolean logics, which already can solve formulas of impressive size - up to 100s of thousands of variables. The main challenge is to find a representation of these formulas that lends itself to making predictions in a scalable way. For challenging problems, the heuristic learned through our approach reduces execution time by a factor of 10 compared to the existing handwritten heuristics.

Keywords: reinforcement learning, deep learning, logics, formal methods, automated reasoning, backtracking search, satisfiability, quantified Boolean formulas

TL;DR: RL finds better heuristics for automated reasoning algorithms.