Re3MHQA: Retrieve, Remove, and Return facts in multi-hop QA

Ziyi Cao, Yunhe Xie, Bingquan Liu, Kun Bu

Abstract: Answering multi-hop questions requires gathering multiple facts, usually over multiple iterations. During this process, retrieving invalid information can disrupt subsequent retrieval, and after collecting necessary information, unnecessary hops may still be made. Furthermore, fact retrieval, error recognition, and stop hop determination are independent functions, each of which requires at least one model, resulting in a waste of GPU resources. To address these challenges, we propose Re3MHQA, a framework designed to Retrieve, Remove, and Return facts in multi-hop Question Answering (QA). Specifically, Retrieve is a dual-encoder-like information retriever; Remove excludes the latest retrieved fact from further retrieval processes if deemed invalid; Return checks whether to conclude the process and returns all gathered facts. Both Remove and Return are based on the cross-encoder strategy. Re3MHQA can significantly enhance the retrieval process with the additional support provided by extra components Remove and Return, unlike methods focusing on improving retrieval performance. Notably, all three components in Re3MHQA are based on one Pre-trained Language Model (PLM), improving the reusability of PLM. Validation on the QASC and MultiRC datasets demonstrates that the Remove and Return components improve F1 performance while obtaining the same number of facts. Further analysis reveals that one PLM can utilize both dual-encoder and cross-encoder strategies for retrieval and text classification, which maintains the performance of individual functions and significantly reduces GPU memory usage.