AutoGeTS: Knowledge-based Automated Generation of Text Synthetics for Improving Text Classification

TMLR Paper6159 Authors

Abstract: When developing text classification models for real-world applications, one major challenge is the difficulty of collecting sufficient data for all text classes. In this work, we address this challenge by utilizing large language models (LLMs) to generate synthetic data and using such data to improve the performance of the models without waiting for more real data to be collected and labeled. As an LLM generates different synthetic data in response to different input examples, we formulate an automated workflow, which searches for input examples that lead to more ``effective'' synthetic data for improving the model concerned. We study three search strategies with an extensive set of experiments, and use experiment results to inform an ensemble algorithm that selects a search strategy according to the characteristics of a class. Our further experiments demonstrate that this ensemble approach is more effective than each individual strategy in our automated workflow for improving classification models using LLMs. The source code of the main software developed for this work is made available at https://anonymous.4open.science/r/AutoGeTS-2B0D.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Serguei_Barannikov1

Submission Number: 6159