One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers Download PDF

Published: 20 Feb 2020, Last Modified: 05 May 2023NeurIPS 2019 Reproducibility Challenge Blind ReportReaders: Everyone
Abstract: The lottery ticket hypothesis states that smaller subnetworks within a larger deep network can be trained in isolation to achieve accuracy similar to that of original network, as long as they are initialized appropriately. However, whether these subnetworks or winning tickets are transferable across datasets and optimizers remains unclear. The paper "One ticket to win them all:generalizing lottery ticket initializations across datasets and optimizers" empirically shows that these winning tickets are transferable. We reproduce the results in the paper from scratch by implementing all the experiments. Our results support the original paper's claim of the winning ticket initializations being transferable. While the paper is replicable, we find that reproducing the paper requires access to large amount of computing resources for generating the winning tickets. Hence, along with the code base, we also open-source the winning tickets we find, so others can avoid the compute-intensive procedure of generating them.
Track: Replicability
NeurIPS Paper Id:
0 Replies