Insertion-based tree decoding

Denis Lukovnikov, Asja Fischer

Abstract:

Sequences are typically decoded in a left-
to-right fashion, requiring as many decoding
steps as there are tokens in the sequence.
Recently, several works have proposed non-
autoregressive decoders that are sub-linear, al-
lowing to decode a sequence using fewer de-
coding steps than the length of the sequence,
and thus substantially speed up inference. In
contrast, non-autoregressive decoding of trees
is less well-analysed, even though trees are
used in important applications like seman-
tic parsing and code generation. In this
work, we present a novel general-purpose par-
tially autoregressive tree decoder that uses tree-
based insertion operations to generate trees in
sub-linear time. We evaluate our approach
on semantic parsing and compare it against
strong baselines, including an insertion-based
sequence decoder. The results demonstrate
that the partially autoregressive tree decoder
reaches competitive accuracies while clearly
reducing the number of decoding steps.