Distilled Pruning: Using Synthetic Data to Win the Lottery

Luke McDermott, Daniel Cummings

TL;DR: This work introduces a novel approach to pruning deep learning models by using distilled data

Keywords: Neural Network Pruning, Data Distillation, Model Compression

Optional Submission Checklist: Yes

Broader Impact Statement: Yes

Paper Availability And License: Yes

Code Of Conduct: Yes

Steps For Environmental Footprint Reduction During Development: Distilled Pruning is inherently designed to reduce the cost of retraining for pruning development by using a smaller, synthetic dataset.

Cpu Hours: 0

Gpu Hours: 150

Tpu Hours: 0

Evaluation Metrics: Yes

Estimated Co2e Footprint: 6.3

Submission Number: 8