QASE Enhanced PLMs: Improved Control in Text Generation for MRC

Anonymous

TL;DR: We develop QASE, a lightweight module that improves fine-tuned generative PLMs' quality and factual consistency on MRC tasks, matching SOTA extractive methods and surpassing GPT-4.

Abstract: To address the challenges of out-of-control generation in generative models for machine reading comprehension (MRC), we introduce the Question-Attended Span Extraction (QASE) module. Integrated during the fine-tuning of pre-trained generative language models (PLMs), QASE enables these PLMs to match SOTA extractive methods and outperform leading LLMs like GPT-4 in MRC tasks, without significant increases in computational costs.

Paper Type: short

Research Area: Question Answering

Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models

Languages Studied: English