Hypothetical Training for Robust Machine Reading Comprehension of Tabular Context

Moxin Li, Wenjie Wang, Fuli Feng, Hanwang Zhang, Qifan Wang, Tat-Seng Chua

Abstract: Machine Reading Comprehension (MRC) models easily learn the spurious correlations from complex context such as tabular data. Counterfactual training—using the original and augmented data—has become a promising solution. However, it is costly to construct faithful counterfactual examples because it is tricky to maintain the consistency and dependency of the table entries. In this paper, we take a more economic fashion to ask hypothetical questions, e.g., “in which year would the net profit be larger if the revenue in 2019 were $38,298?”, whose effects on the answers are equivalent to those expensive counterfactual tables. We propose a hypothetical training framework that uses pairs of examples with different hypothetical questions to supervise the direction of model gradient w.r.t. the input towards the answer change. We conduct experiments on MRC datasets with factual and hypothetical examples. Performance gain on a newly constructed stress test validates the effectiveness and rationality of our approach.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)