When Do Decompositions Help for Machine Reading?

Kangda Wei, Dawn Lawrie, Benjamin Van Durme, Yunmo Chen, Orion Weller

Submission Type: Regular Short Paper

Submission Track: Question Answering

Keywords: decomposition, machine reading, question answering

TL;DR: Decompositions are only helpful for machine reading when you have a small amount of labeled data; otherwise learn the decomposition implicitly through fine-tuning

Abstract: Answering complex questions often requires multi-step reasoning in order to obtain the final answer. Most research into decompositions of complex questions involves open-domain systems, which have shown success in using these decompositions for improved retrieval. In the machine reading setting, however, work to understand when decompositions are helpful is understudied. We conduct experiments on decompositions in machine reading to unify recent work in this space, using a range of models and datasets. We find that decompositions can be helpful in zero or limited-data settings, giving several points of improvement in exact match. However, we also show that when models are given access to around a few hundred or more examples, decompositions are not helpful (and can actually be detrimental). Thus, our analysis implies that models can learn decompositions implicitly even with limited data.

Submission Number: 1974