Beyond the Zero-Crossing: Finding the Optimal Polarity Threshold for the Sign Hypothesis

NeurIPS 2025 Workshop NeurReps Submission169 Authors

Keywords: Lottery Ticket Hypothesis, Sign Hypothesis, Network Pruning

TL;DR: We study perturbations to the sign hypothesis for the LTH and get some new findings regarding the centre of polarity

Abstract: A central tenet of the Lottery Ticket Hypothesis is that the initial signs of weights are a critical ingredient for success. This "Sign Hypothesis" has always assumed that a weight's polarity is determined by its relationship to zero. We challenge this long-held assumption by introducing a perturbative analysis centered on a "phi-center" (φ_center) parameter, which establishes a learnable, non-zero threshold for polarity. Our comprehensive experiments show that network performance consistently peaks at a non-zero φ_center, with its optimal value varying with task complexity. This reveals a "complexity gradient": the polarity definition is irrelevant for simple tasks (MNIST) but becomes paramount for complex ones (CIFAR-10/100). These results offer a more nuanced explanation for the success of magnitude pruning: it effectively removes weights whose initial signs are noisy and misleading because they are evaluated against a suboptimal, zero-centered threshold. To the best of our knowledge, this is the first work to decouple weight polarity from the zero-crossing for lottery ticket pruning. This provides a new direction for understanding initialization and sparsity.

Submission Number: 169