Structural Pruning of Large Language Models via Neural Architecture Search

Published: 06 Jul 2023, Last Modified: 15 Aug 2023AutoML 2023 (Workshop)EveryoneRevisionsBibTeX
TL;DR: We propose multi-objective weight-sharing based neural-architecture search to prune large language models when fine-tuned on downstream tasks.
Keywords: neural architecture search, large language models
Optional Submission Checklist: Yes
Broader Impact Statement: Yes
Paper Availability And License: Yes
Code Of Conduct: Yes
Cpu Hours: 0
Gpu Hours: 0
Tpu Hours: 0
Submission Number: 5