Complex program induction for querying knowledge bases in the absence of gold programs

Ghulam Ahmed Ansari, Amrita Saha, Karthik Sankaranarayanan, Soumen Chakrabarti, Abhishek Laddha

Abstract:

Recent years have seen increasingly complex question-answering on knowledge bases
(KBQA) involving logical, quantitative, and
comparative reasoning over KB subgraphs.
Neural Program Induction (NPI) is a pragmatic
approach toward modularizing the reasoning
process by translating a complex natural language query into a multi-step executable program. While NPI has been commonly trained
with the ‘‘gold’’ program or its sketch, for
realistic KBQA applications such gold programs are expensive to obtain. There, practically only natural language queries and the
corresponding answers can be provided for
training. The resulting combinatorial explosion in program space, along with extremely
sparse rewards, makes NPI for KBQA ambitious and challenging. We present Complex
Imperative Program Induction from Terminal
Rewards (CIPITR), an advanced neural programmer that mitigates reward sparsity with
auxiliary rewards, and restricts the program
space to semantically correct programs using
high-level constraints, KB schema, and inferred answer type. CIPITR solves complex
KBQA considerably more accurately than
key-value memory networks and neural symbolic machines (NSM). For moderately complex queries requiring 2- to 5-step programs,
CIPITR scores at least 3× higher F1 than the
competing systems. On one of the hardest class
of programs (comparative reasoning) with
5–10 steps, CIPITR outperforms NSM by a
factor of 89 and memory networks by 9 times.