Neural models combining representation learning and reasoning in an end-to-end trainable manner are receiving increasing interest. However, their use is severely limited by their computational complexity, which renders them unusable on real world datasets. We focus on the Neural Theorem Prover model proposed by Rocktäschel and Riedel (2017), a continuous relaxation of the Prolog backward chaining algorithm where unification between terms is replaced by the similarity between their embedding representations. For answering a given query, this model needs to consider all possible proof paths, and then aggregate results -- this quickly becomes infeasible even for small Knowledge Bases. We observe that we can accurately approximate the inference process in this model by considering only proof paths associated with the highest proof scores. This enables inference and learning on previously impracticable Knowledge Bases.

Towards Neural Theorem Proving at Scale

The task of translating between programming languages differs from the challenge of translating natural languages in that programming languages are designed with a far more rigid set of structural and grammatical rules. Previous work has used a tree-to-tree encoder/decoder model to take advantage of the inherent tree structure of programs during translation.  Neural decoders, however, by default do not exploit known grammar rules of the target language.  In this paper, we describe a tree decoder that leverages knowledge of a language's grammar rules to exclusively generate syntactically correct programs.  We find that this grammar-based tree-to-tree model outperforms the state of the art tree-to-tree model in translating between two programming languages on a previously used synthetic task.

Program Language Translation Using a Grammar-Driven Tree-to-Tree Model

Deep reinforcement learning has led to many recent breakthroughs, but the learnt policies are often based on black-box neural networks, which makes them difficult to interpret and to impose desired specification constraints during learning. We present an iterative framework, MORL, for improving the learned policies using program synthesis. Concretely, we propose to use synthesis techniques to obtain a symbolic representation of the learned policy, which can then be debugged manually or automatically using program repair. After the repair step, we use behavior cloning to obtain the policy corresponding to the repaired program, which is then further improved using gradient descent. This process continues until the learned policy satisfies desired constraints. We instantiate MORL for the simple CartPole problem and show that the programmatic representation allows for high-level modifications that in turn lead to improved learning of the policies. 


Towards Mixed Optimization forReinforcement Learning with Program Synthesis

        We present a {\em neurosymbolic approach} to the lifelong learning
        of algorithmic tasks that mix perception and procedural
        reasoning. Reusing high-level concepts across domains and learning
        complex procedures are key challenges in lifelong learning. We
        show that a combination of gradient-based learning and {symbolic
        program synthesis} can be a more effective response to these
        challenges than purely neural methods. Our approach,
        called \system, represents neural networks as strongly typed,
        end-to-end differentiable functional programs that use 
        symbolic higher-order combinators to compose a library of neural
        functions. 
        Our learning algorithm consists of: (1) a symbolic program
        synthesizer that performs a type-directed search over
        parameterized programs, and decides on the 
        library functions to
        reuse, and the architectures to combine them, while
        learning a sequence of tasks; and (2) a neural module that
        trains these programs using stochastic gradient descent.
        Our experiments show that \system transfers high-level
        concepts more effectively than traditional transfer learning
        and progressive neural networks.


Synthesis of Differentiable Functional Programs for Lifelong Learning

We present a simple approach for synthesizing code from keywords. Our system takes keywords, a subset of tokens from the target code, and generates a line of code. The main challenge is that we only have the output of the system and do not know the real input distribution. We construct simple synthetic distributions by dropping each keyword randomly and based on its frequency, which still generalize to various test distributions. We train a standard sequence-to-sequence model on the synthetic training data and successfully synthesize 71.5% of examples in our manually generated test set. Our approach is simple and language-agnostic and therefore can be easily extended and applied to any programming language.

Autocompletion of Code from Keywords

We present a program synthesis-oriented dataset consisting of human written problem statements and solutions for these problems.  The problem statements were collected via crowdsourcing and the program solutions were extracted from human-written solutions in programming competitions, accompanied by input/output examples. We propose using this dataset for the program synthesis tasks aimed at working with real user-generated data. As a baseline, we present few models, with the best model achieving 5.6% accuracy, showcasing both complexity of the dataset and large room for future research.

NAPS: Natural Program Synthesis Dataset

We present a neural semantic parser that translatesnatural language questions intoexecutableSQLqueries with two key ideas. First, we develop anencoder-decoder model, where the decoder usesa simple type system of SQL to constraint theoutput prediction, and propose a value-based losswhen copying from input tokens. Second, we ex-plore using the execution semantics of SQL to re-pair decoded programs that result in runtime erroror return empty result.  We propose two model-agnostics repair approaches, an ensemble modeland a local program repair, and demonstrate theireffectiveness over the original model. We evalu-ate our model on the WikiSQL dataset and showthat our model achieves close to state-of-the-artresults with lesser model complexity.

Execution-Guided Neural Program Decoding

We consider learning to generalize and extrapolate with limited data to harder compositional problems than a learner has previously seen. We take steps toward this challenge by presenting a characterization, algorithm, and implementation of a learner that programs itself automatically to reflect the structure of the problem it faces. Our key ideas are (1) transforming representations with modular units of computation is a solution for decomposing problems in a way that reflects their subproblem structure; (2) learning the structure of a computation can be formulated as a sequential decision-making problem. Experiments on solving various multilingual arithmetic problems demonstrate that our method generalizes out of distribution to unseen problem classes and extrapolates to harder versions of the same problem. Our paper provides the first element of a framework for learning general-purpose, compositional and recursive programs that design themselves.

Automatically Constructing Compositional and Recursive Learners

Algorithm learning is a core problem in artificial
intelligence with significant implications on automation
level that can be achieved by machines.
Recently deep learning methods are emerging for
synthesizing an algorithm from its input-output
examples, the most successful being the Neural
GPU, capable of learning multiplication. We
present several improvements to the Neural GPU
that substantially reduces training time and improves
generalization. We introduce a new technique
- hard nonlinearities with saturation costs -
that has general applicability. We also introduce
a technique of diagonal gates that can be applied
to active-memory models. The proposed architecture
is the first capable of learning decimal
multiplication end-to-end.

ICML 2018 Workshop NAMPI Submissions