Data-to-text Generation with Verification and Correction Prompting

Anonymous

TL;DR: Applying a feedback system to small models to reduce semantic errors in data-to-text generation.

Abstract:

Small language models like T5 excel in generating high-quality text for data-to-text tasks, offering adaptability and cost-efficiency compared to Large Language Models (LLMs). However, they frequently miss keywords, which is considered one of the most severe and common errors in this task.

In this work, we explore the potential of using feedback systems to enhance semantic fidelity in smaller language models for data-to-text generation tasks, through our Verification and Correction Prompting (VCP) approach.

In the inference stage, our approach involves a multi-step process, including generation, verification, and regeneration stages. During the verification stage, we implement a simple rule to check for the presence of every keyword in the prediction. Recognizing that this rule can be inaccurate, we have developed a carefully designed training procedure, which enabling the model to incorporate feedback from the error-indication prompt effectively, despite its potential inaccuracies.

The VCP approach effectively reduces the Semantic Error Rate (SER) while maintaining the text's quality.

Paper Type: long

Research Area: Generation

Contribution Types: NLP engineering experiment

Languages Studied: English