Learn to Synthesize Compact Datasets by Matching Effects
Keywords: Deep Learning, Dataset Distillation
TL;DR: Deep Learning,AIGC
Abstract: The emerging field of data distillation aims to compress large datasets by aligning synthetic and real data representations to create a highly informative dataset. The optimization objectives of data distillation focus on aligning representations by using process alignment methods such as trajectory and gradient matching. However, this approach is limited by the strict alignment of intermediate quantities between synthetic and real data and the mismatch between their optimization trajectories. To address these limitations, a new data distillation method called effect alignment is proposed, which aims to only push for the consistency of endpoint training results. The approach uses classification tasks to estimate the impact of replacing real training samples with synthetic data, which helps to learn a synthetic dataset that can replace the real dataset and achieve effect alignment. The method is efficient and does not require costly mechanisms, and satisfactory results have been achieved through experiments.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1268
Loading