How do we answer complex questions: Discourse structure of long form answers

Anonymous

Abstract: Long form answers, consisting of multiple sentences, can provide nuanced and comprehensive answers to a broader set of questions. However, little prior work exists on this task. To better understand this complex task, we study the functional structure of long form answers on two datasets, Natural Questions~\cite{kwiatkowski2019natural} and ELI5~\cite{Fan2019ELI5LF}. Our main goal is to understand how humans organize information to craft complex answers. We develop an ontology of sentence-level functional roles for long form answers, and annotate 3.3k sentences in 542 examples. Our annotated data enables training a reliable role classifier that can be used for automatic analysis and thus reveals machine generated answers are structured worse than human written answers. Our data further yields an extractive summarization dataset for long form answers, giving models the ability to identify a concise answer to a complex query.