Hierarchical Attention Decoder for Solving Math Word Problems

Anonymous

Abstract: To answer math word problems (MWPs), models need to formalize equations from the source text of math problems. Recently, the tree-structured decoder has significantly improved model performance on this task by generating the target equation in a tree format. However, current decoders usually ignore the hierarchical relationships between tree nodes and their parents, which hinders further improvement. Thus, we propose a structure called hierarchical attention tree to aid the generation procedure of the decoder. As our decoder follows a graph-based encoder, our full model is therefore named as Graph to Hierarchical Attention Tree (G2HAT). We show a tree-structured decoder with hierarchical accumulative multi-head attention leads to significant performance improvement and reaches new state-of-the-art (SOTA) on both English MAWPS and Chinese Math23k MWP benchmarks. For further study, we also apply pre-trained language models for G2HAT, which even results in new higher performance.