Synthesized Differentiable Programs

Lucas Paul Saldyt

Keywords: program synthesis, neural compilation, neurosymbolic, abstract, software, algorithm, machine learning, neural networks

TL;DR: We introduce and evaluate a combined program synthesis algorithm that uses neural compilation to enable gradient optimization.

Abstract: Program synthesis algorithms produce interpretable and generalizable code that captures input data but are not directly amenable to continuous optimization using gradient descent.In theory, any program can be represented in a Turing complete neural network model, which implies that it is possible to compile syntactic programs into the weights of a neural network by using a technique known as \textit{neural compilation}.This paper presents a combined algorithm for synthesizing syntactic programs, compiling them into the weights of a neural network, and then tuning the resulting model. This paper's experiments establish that program synthesis, neural compilation, and differentiable optimization together form an efficient algorithm for inducing abstract algorithmic structure and a corresponding local set of desirable complex programs