Data Synthesis with Gradient Inversion

Ishika Agarwal, Mayank Shrivastava, Dilek Hakkani-Tür

Keywords: data synthesis

TL;DR: We use gradient inversion to guide synthetic data generation.

Abstract: Data synthesis pipelines are costly and wasteful: current approaches generate large pools of candidates via rejection sampling, only to discard the majority as incorrect, redundant, or harmful. This work explores two core questions: (1) how can we identify the specific areas where a model is weak, and (2) how can we synthesize targeted data in those areas with greater reliability? We address these by designing a data synthesis methodology grounded in gradient inversion—an attack strategy originally used to recover training inputs from observed gradients. Concretely, we train a modified encoder-decoder that takes a language model's gradient as input, projects it into a latent space, and decodes it into text. By targeting gradients associated with poor model performance (answering the first question), our pipeline generates new training examples precisely in the regions where the model struggles (answering the second question), improving both the efficiency and relevance of synthetic data generation. As this is a work-in-progress, we present initial findings: while we are able to reconstruct text that retains general topics of the original text, the generation is slightly noisy.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 50