Step-by-Step Mastery: Enhancing Universal Constraint Following Ability of Large Language Models

ACL ARR 2024 December Submission2279 Authors

Abstract: It is crucial for large language models (LLMs) to follow instructions that involve multiple constraints. However, many soft constraints are semantically related and difficult to verify through automated methods. These constraints remain a significant challenge for LLMs. To enhance the ability of LLMs to follow soft constraints, we initially design a pipeline to obtain high-quality outputs automatically. Additionally, to fully utilize the acquired data, we introduce a progressive training paradigm based on curriculum learning. We experimentally evaluate the effectiveness of our methods in improving LLMs' soft constraint following ability and analyze the factors driving the improvements. To support further research, we will release the code and data associated with this study.

Paper Type: Long

Research Area: Language Modeling

Research Area Keywords: fine-tuning,continual learning

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 2279