Self–Boost: Boosting LLMs with Iterative Self-Generated Data
Abstract: Instruction finetuned large language models (LLMs) have shown impressive performance solving a diverse range of natural language processing (NLP) tasks involving classification and reasoning. However, this can be particularly challenging in low-data regimes. Recent methods have shown boosting via iterative full finetuning to be an effective method to augment the training data by using the incorrect examples to generate synthetic data using a teacher LLM. However, data generation at scale using a teacher LLM can be costly, and full finetuning can be computationally expensive. To address this, we introduce Self–Boost, an iterative data augmentation and instruction finetuning strategy that has no external dependence on any teacher models. Self–Boost uses parameter efficient finetuning (PEFT) with Llama 3 8B to instruction finetune a model using the seed data, uses the same model to generate examples similar to the misclassifications, and also the same model to verify and filter the generated examples. Our experiments show that performance on TREC, GSM8K, and CaseHOLD improves by 21.6\%, 5.6\% and 1.3\% respectively, when compared to our baseline.
Paper Type: Short
Research Area: Efficient/Low-Resource Methods for NLP
Research Area Keywords: Efficient/Low-Resource Methods for NLP, Machine Learning for NLP
Contribution Types: Approaches to low-resource settings
Languages Studied: English
Submission Number: 4438
Loading