- TL;DR: We introduce Programming Puzzles, an expressive set of reasoning problems that require minimal understanding of other domains like language. We then introduce TroubleMakers, an algorithm to generate hard programming puzzles.
- Abstract: AI today is far from being able to write complex programs. What type of problems would be best for computers to learn to program, and how should such problems be generated? To answer the first question, we suggest programming puzzles as a domain for teaching computers programming. A programming puzzle consists of a short program for a Boolean function f(x) and the goal is, given the source code, to find an input that makes f return True. Puzzles are objective in that one can easily test the correctness of a given solution x by seeing whether it satisfies f, unlike the most common representations for program synthesis: given input-output pairs or an English problem description, the correctness of a given solution is not determined and is debatable. To address the second question of automatic puzzle generation, we suggest a GAN-like generation algorithm called “Troublemaker” which can generate puzzles targeted at any given puzzle-solver. The main innovation is that it adapts to one or more given puzzle-solvers: rather than generating a single dataset of puzzles, Tro
- Keywords: program synthesis, reasoning, math problems