Enhancing Objective Query Distractor Generation in Pre-trained Encoder-Decoder Models via Contrastive Learning

ACL ARR 2025 February Submission6029 Authors

Abstract:

Distractor generation is a critical task in objective types of assessments, including fill-in-the-blank and multiple-choice questions. Recent advances in pre-trained transformer-based models have shown success in generating
distractors. Prior research efforts focus on fine-tuning pre-trained encoder-decoder models with data augmentation strategies to improve this task, but these models often fail to capture the full semantic representation of a given query-answer and related distractors. Data augmentation methods
often rely on expanding the quantity of proposed distractors, which can introduce noise into the models without necessarily enhancing its understanding of the deeper semantic relationships between distractors. This paper introduces a novel distractor generation model based on contrastive learning to capture semantic details from the query-answer and distractor sequence encodings. The contrastive learning method trains the model to recognize essential semantic features, necessary to generate in-context distractors. The extensive experiments on two public datasets indicate that contrastive learning is essential in encoder-decoder models. It
significantly outperforms baseline models and advances the NDCG@3 score from 24.68 to 32.33 in the MCQ dataset and 26.66 to 36.68 in the SciQ dataset.

Paper Type: Long

Research Area: Generation

Research Area Keywords: Generation, NLP Applications

Contribution Types: NLP engineering experiment

Languages Studied: T5, BART, GPT-3

Submission Number: 6029