Complex QA with Diverse Knowledge Sources: Novel Benchmark and Approach
Primary Area: datasets and benchmarks
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Question Answering Dataset, Natural Language Processing
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: The complex question-answering task has traditionally revolved around breaking down multi-hop questions into simpler, single-hop queries and then addressing each of these simplified questions by extracting relevant information from an external knowledge source. Recent advancements in research have taken this a step further, focusing on the challenging scenario where multi-hop questions require evidence from a variety of sources, including unstructured text from sources like Wikipedia, as well as structured knowledge bases and tables. However, in many cases, structured knowledge sources have been treated as supplementary, with the primary emphasis on retrieving information from unstructured text, which often dictates the overall performance of the system. In this research paper, we explore the performance of state-of-the-art models in complex QA tasks when both structured and unstructured knowledge retrieval are given equal importance. Recognizing the absence of a well-established benchmark for such scenarios, we have curated a QA dataset that specifically requires structured knowledge retrieval to obtain accurate answers, as relying solely on unstructured retrieval is insufficient to cover all aspects of these complex questions. Furthermore, we have developed tools that employ symbolic language generation to facilitate information retrieval from structured knowledge sources. Our experiments, conducted on our newly created dataset, highlight the effectiveness and efficiency of the proposed architecture in handling these intricate question-answering tasks.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5707
Loading