More Semantically Focused Modeling for Semantic Text Matching

Anonymous

Abstract: As is widely acknowledged, Pre-trained Language Models (PLMs) acquire the capability to encode deep sentence semantics through pre-training. Semantic Text Matching (STM) task has greatly benefited from this capacity. However, the extent to which PLMs can fully exploit semantic encoding, rather than merely relying on some superficial pattern recognition in this task, remains a matter for investigation. We argue that a model’s ability to provide consistent judgments despite variations in phrasing indicates its reliance on semantic interpretation. Based on this perspective, we investigate the extent to which the model captures semantics and introduce a novel training architecture aimed at enhancing the semantic modeling capacity of PLMs in STM tasks. Our approach is validated through rigorous experimentation on four benchmark datasets: LCQMC, BQ, QQP, and MRPC, where we achieve state-of-the-art performance on three of them.

Paper Type: short

Research Area: Semantics: Sentence-level Semantics, Textual Inference and Other areas

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: Chinese,English