Multi-Attribute Constraint Satisfaction via Language Model Rewriting
Abstract: Obeying precise constraints on top of multiple external attributes is a common computational problem underlying seemingly different domains, from controlled text generation to protein engineering. Existing language model (LM) controllability methods for multi-attribute constraint satisfaction often rely on specialized architectures or gradient-based classifiers, limiting their flexibility to work with arbitrary black-box evaluators and pretrained models. Current general-purpose large language models, while capable, cannot achieve fine-grained multi-attribute control over external attributes. Thus, we create Multi-Attribute Constraint Satisfaction (MACS), a generalized method capable of finetuning language models on any sequential domain to satisfy user-specified constraints on multiple external real-value attributes. Our method trains LMs as editors by sampling diverse multi-attribute edit pairs from an initial set of paraphrased outputs. During inference, LM iteratively improves upon its previous solution to satisfy constraints for all attributes by leveraging our designed constraint satisfaction reward. We additionally experiment with reward-weighted behavior cloning to further improve the constraint satisfaction rate of LMs. To evaluate our approach, we present a new Fine-grained Constraint Satisfaction (FineCS) benchmark, featuring two challenging tasks: (1) Text Style Transfer, where the goal is to simultaneously modify the sentiment and complexity of reviews, and (2) Protein Design, focusing on modulating fluorescence and stability of Green Fluorescent Proteins (GFP). Our empirical results show that MACS achieves the highest threshold satisfaction in both FineCS tasks, outperforming strong domain-specific baselines. Our work opens new avenues for generalized and real-value multi-attribute control, with implications for diverse applications spanning natural language processing and bioinformatics.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: - Added statistical significance tests as recommended by reviewer VzBx. Updated the conclusion about best strategies with a more nuanced analysis based on the updated results.
- Included presentation clarities and algorithm box to provide a more formal description of the core method.
- Following reviewer JCoR's suggestion, modified the abstract and introduction to highlight the distinction of real-value multi-attribute constraints addressed in the paper (different from categorical and lexical constraints that are left for future work).
Assigned Action Editor: ~Karthik_R_Narasimhan1
Submission Number: 2902
Loading