RDR: The Recap, Deliberate, and Respond Method for Enhanced Language Understanding
Keywords: Natural Language Understanding, Knowledge Graphs, Commonsense Reasoning, Logical Reasoning, External Knowledge Fusion, Embedding, Sequential Learning, Attention
TL;DR: An Empirical Methodology to extract, acquire and fuse relevant Commonsense & Logical relationship based External Knowledge for enhanced Natural Language Understanding
Abstract: Natural language understanding (NLU) using neural network pipelines often requires additional context that is not solely present in the input data, such as external knowledge graphs. Through prior research, it has been evident that NLU benchmarks are susceptible to manipulation by neural models - these models exploit statistical artifacts within the encoded external knowledge to artificially inflate performance metrics for downstream tasks. Our proposed approach, known as the Recap, Deliberate, and Respond (RDR) paradigm, addresses this issue by incorporating three distinct objectives within the neural network pipeline. The Recap objective involves paraphrasing the input text using a paraphrasing model in order to summarize and encapsulate salient information of the input. Deliberate refers to encoding the external graph information that is relevant to entities in the input text using a graph embedding model. Finally, Respond employs a classification head model that integrates representations from the Recap and Deliberate steps to generate the final prediction. By cascading these three models and minimizing a combined loss, we mitigate the potential of the model gaming the benchmark, while establishing a robust method for capturing the underlying semantic patterns to achieve accurate predictions. We conduct tests on multiple GLUE benchmark tasks to evaluate the effectiveness of the RDR method. Our results demonstrate improved performance compared with competitive baselines, with an enhancement of up to 2% on standard evaluation metrics. Furthermore, we analyze the observed behavior of semantic understanding of the RDR models, emphasizing their ability to avoid gaming the benchmark while accurately capturing the true underlying semantic patterns.
Submission Number: 19
Loading